Find answers from the community

Updated 3 months ago

Hi is there a way I can predict how many

Hi, is there a way I can predict how many API calls a given query is going to use before I send it? I am presently watching a list index / accumulate query go over 214 nodes and it is taking a very long time (hours) and has used more than 214 api calls (which I was not expecting and worries me).

2 comments

LLogan M

A list index will always query every node in the index.

You can predict token usage using a mock LLM and the token counting callback

https://gpt-index.readthedocs.io/en/latest/core_modules/supporting_modules/cost_analysis/usage_pattern.html

LLogan M

Sounds like you probably didn't want to use a list index 😅

Add a reply