Find answers from the community

Updated 2 months ago

hi 👋. I'm having issues with claude 3

hi 👋. I'm having issues with claude 3 streaming. I seem to only be getting the first delta. this is with a variety of engines, but can be replicated with SimpleChatEngine. I'm on the latest version of llama-index (0.10.18).

4 comments

BBrandon

when I disable streaming, I get complete responses. when I enable streaming, I get incomplete responses, e.g. "I apolog" instead of "I apologize, I am Claude."

LLogan M

How are you streaming?

LLogan M

Like, a quick code sample of your setup?

BBrandon

using astream_chat and async for text in response.async_response_gen():

I printed some stuff out in the Anthropic class itself. I can get that to you in a bit.

Add a reply