Streaming Chat Response with FastAPI

At a glance

The community member has a FastAPI endpoint that streams response data, but they are receiving the data in a format they don't want (data: data=' R' source='internal'). The community members suggest converting the pydantic objects to dicts or JSON strings before returning the response. One community member provides a specific solution using async def gen(): async for ev in handler.stream_events(): yield ev.model_dump() return EventSourceResponse(gen()). However, there is no explicitly marked answer in the comments.

lllm_dev

i have fastapi like this

@r.get("/chat", tags=["basic"])
async def get_prompt(request: RmsGptRequest = Depends()):
logger.info(f"Getting results with params: {request}")

# Stream the response
handler = tara.run(query=request.prompt,user_email=request.user_email)
return EventSourceResponse(handler.stream_events())

how can i return the stream data as json directly from workflow ?? because I'm getting the data like this in Postman

data: data=' R' source='internal'

3 comments

LLogan M

The events are pydantic objects. Convert them to dicts or json string

LLogan M

Plain Text

async def gen():
  async for ev in handler.stream_events():
    yield ev.model_dump()
return EventSourceResponse(gen())

lllm_dev

ooo, thx @Logan M appreciate it. :LlamaIndex:

Add a reply

Find answers from the community

Streaming Chat Response with FastAPI