Find answers from the community

Updated 2 months ago

hi πŸ‘‹. I'm having issues with claude 3

hi πŸ‘‹. I'm having issues with claude 3 streaming. I seem to only be getting the first delta. this is with a variety of engines, but can be replicated with SimpleChatEngine. I'm on the latest version of llama-index (0.10.18).
B
L
4 comments
when I disable streaming, I get complete responses. when I enable streaming, I get incomplete responses, e.g. "I apolog" instead of "I apologize, I am Claude."
How are you streaming?
Like, a quick code sample of your setup?
using astream_chat and async for text in response.async_response_gen():

I printed some stuff out in the Anthropic class itself. I can get that to you in a bit.
Add a reply
Sign up and join the conversation on Discord