Find answers from the community

Updated last month

Tool cals

Hey, so how would i manage to estimate costs for my RAG pipeline if i use the OpenAIAgentRunner?
Are tool calls billed differently then completion calls? And also how do i have to calculate the context?
What i mean here is that for example if i have a chunk size of 512 and retrieve 30 nodes, do i have to calculate 512*30+len(tokenize(message)) as average token count per message?
L
3 comments
Tools are counted as input tokens. OpenAI doesn't make this easy, but you can roughly estimate using the token counts of tool.metadata.to_openai_tool() output
So in your example, the input token estimation would be `512ร—30 + len(chat_history) + len(tools)
Add a reply
Sign up and join the conversation on Discord