Hey there,

Hey there,
I hope you're doing well. I wanted to touch base regarding a little snag I've hit while working on our app.

Currently, I am working on an application where I have been utilizing the streaming feature. However, I have run into a significant hurdle: I cannot seem to get the streaming functionality to work as expected. Specifically, I am struggling to ensure that data streaming occurs correctly.

I have used other applications with similar versions where the streaming feature has worked flawlessly. However, in this particular application, I am unable to get it to function as expected.

To provide you with more context, I am employing RouterRetriever, a synthesizer, and a RetrieverQueryEngine in my implementation. I have configured the necessary attributes for data streaming (streaming=True), but unfortunately, it has not yielded the desired results. I have even tried the implementation in both Google Colab and Hugging Face, but I continue to face the same issue.

llama-index==0.10.20

Below is how I have implemented the function in my code:

response_synthesizer = get_response_synthesizer(text_qa_template=qa_prompt, streaming=True)

query_engine = RetrieverQueryEngine.from_args(
retriever=custom_retriever,
response_synthesizer=response_synthesizer,
node_postprocessors=[rerank],
streaming=True
)

respuesta = query_engine.query(pregunta)
respuesta.print_response_stream() # No funciona

I have also attempted to implement it this way in Hugging Face:

def responder(pregunta):
respuesta = query_engine.query(pregunta)

partial_message = ""
for text in respuesta.response_gen:
partial_message += text
yield partial_message # and it still doesn't work

I would greatly appreciate it if you could provide me with any guidance or advice on how to resolve this issue. I am willing to provide any additional information you may need to assist me in resolving this situation.

Appreciate your help!