Find answers from the community

s
F
Y
a
P
Updated last year

I have developed a rag system now i want

I have developed a rag system now i want to integrate it on my website as a chatbot. What are the ways in which i can do that for example - i am thinking of creating an API in django but problem is how can i return the response in a streaming mode
L
a
l
8 comments
I would use fastapi, it's pretty easy to return streaming responses from query engines, I think you just need the generator
@Logan M What do you mean by generator
Like, the streaming response is called a generator

response.response_gen

So you iterate over that to get each token

for token in response.response_gen
@Logan M i have implement this on Fastapi plz provide ur feedback
Attachment
rn_image_picker_lib_temp_dc34818e-2465-4a08-9fd2-1ceb3f5eaf74.jpg
seems to make sense. Any specific reason for using deepcopy?
I am getting this type of response how can i remove data
2)It is possible to get a streaming response like we get in chatgpt
Attachment
rn_image_picker_lib_temp_7cc00bcc-0709-4f42-bfe9-9042acbd4efd.jpg
Add a reply
Sign up and join the conversation on Discord