Find answers from the community

Updated 2 months ago

Langchain

Hello everyone.
I've upgraded llama index to the latest yesterday, moving from the 0.6.38 to 0.7.2.
I've migrated the code and everything seems to work except the langchain tooken counter and costs.

Plain Text
from langchain.callbacks import get_openai_callback
with get_openai_callback() as cb:
   ...../// llm stuff 
   print(cb.total_tokens)


Until yesterday this code was working fine, but from today I got always 0 as token counters and costs.
I'm currently using this as underlying LLM connector.
Plain Text
from llama_index.llms import AzureOpenAI


Did something change around it?
L
1 comment
Our azure LLM is not using langchain, so the langchain token counter likely won't work

You can go back to using AzureChatOpenAI/AzureOpenAI from langchain, or you can use our token counting

https://gpt-index.readthedocs.io/en/latest/examples/callbacks/TokenCountingHandler.html
Add a reply
Sign up and join the conversation on Discord