Find answers from the community

Updated last year

I have developed a rag system now i want

I have developed a rag system now i want to integrate it on my website as a chatbot. What are the ways in which i can do that for example - i am thinking of creating an API in django but problem is how can i return the response in a streaming mode

8 comments

LLogan M

I would use fastapi, it's pretty easy to return streaming responses from query engines, I think you just need the generator

aadeelhasan

@Logan M What do you mean by generator

LLogan M

Like, the streaming response is called a generator

response.response_gen

So you iterate over that to get each token

for token in response.response_gen

llucastonon

Might not be too different with llamaindex
https://indishmarketer.medium.com/streaming-for-langchain-agents-fastapi-6d7ea0effe28

aadeelhasan

@Logan M i have implement this on Fastapi plz provide ur feedback

Attachment