Find answers from the community

Updated 3 months ago

Token usage

Hey Team, how can I pass these values to a response object together with answer:
INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 1221 tokens
INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens

2 comments

LLogan M

I think you'll have to check the last_token_usage of the embed model/llm predictor

index.service_context.llm_predictor.last_token_usage (or embed_model too)

SSimonas

🙏 thanks

Add a reply