Find answers from the community

Home
Members
sylvinbestin
s
sylvinbestin
Offline, last seen 3 months ago
Joined September 25, 2024
hi, does anyone have token counting working for streaming? seems prompt and completion tokens are not tracked. in verbose counter logs it seems prompt and completion events are not raised
16 comments
s
L
i'm using a vector index as chat engine in context mode
43 comments
s
L
has anyone seen "cross talk" for multi user streaming response? i'm seeing when 2 simulatneous streaming chat calls cause the response streams to cross talk.
22 comments
s
L