Find answers from the community

Updated 2 months ago

Response

OpenAi response looks like----
{
"id":"chatcmpl-abc123",
"object":"chat.completion",
"created":1677858242,
"model":"gpt-3.5-turbo-0301",
"usage":{
"prompt_tokens":13,
"completion_tokens":7,
"total_tokens":20
},
"choices":[
{
"message":{
"role":"assistant",
"content":"\n\nThis is a test!"
},
"finish_reason":"stop",
"index":0
}
]
}
I know that there is a new tokenizer implementation in llama-index, but can we get this kind of body straight in the response, or at least the same usage body in the response.
L
S
2 comments
If you used the llm object itself directly, the raw json is available.

What do you need from this json? I remember you wanted the token counts, which are now correctly separated here

https://gpt-index.readthedocs.io/en/latest/examples/callbacks/TokenCountingHandler.html
Thanks, let me try with the new handler.
Add a reply
Sign up and join the conversation on Discord