LlamaIndex
Log in

Find answers from the community

Updated last year

Having an issue with json mode.. I’m

bbig_ol_tender
·
Having an issue with json mode.. I’m getting thousands of new lines from my request with the actual response somewhere in the middle. Anyone seen this?
L
b
12 comments
LLogan M
I have not. That sounds wild though haha
bbig_ol_tender
It is
bbig_ol_tender
Only seems to happen with very short completions
bbig_ol_tender
J/k it’s happening with long completions too
bbig_ol_tender
Okay others are seeing this
bbig_ol_tender
https://community.openai.com/t/json-format-causes-infinite-n-n-n-n-in-response/519333
LLogan M
the ultimate completion token wasting scheme 💸
LLogan M
that really sucks though
LLogan M
glad you aren't the only one
bbig_ol_tender
Yeah it’s pretty upsetting tbh, I prob spend $10 debugging since it was returning so many tokens
bbig_ol_tender
And it’s somewhat nondeterministic so that was making me thing it was on my side
LLogan M
Dang :PSadge: Time to turn off json mode I guess for now
Add a reply
Sign up and join the conversation on Discord