LlamaIndex
Log in

Find answers from the community

Updated last year

Hi! Why is open ai api called many times

At a glance
aaiua
·
Hi! Why is open ai api called many times for one query?
The first response already returns the correct message. And message of second response is also same message.

**
Trace: query
|_CBEventType.QUERY -> 96.195128 seconds
|_CBEventType.RETRIEVE -> 1.351961 seconds
|_CBEventType.EMBEDDING -> 1.326677 seconds
|_CBEventType.SYNTHESIZE -> 94.842815 seconds
|_CBEventType.TEMPLATING -> 4.6e-05 seconds
|_CBEventType.LLM -> 48.121998 seconds
|_CBEventType.TEMPLATING -> 3.5e-05 seconds
|_CBEventType.LLM -> 46.620856 seconds
**
L
a
4 comments
LLogan M
Have you set the top k rather high? Or changed the response mode?
aaiua
Yes, I've set high similarity_top_k.
So I tried to lower the value, and received only one response.

Thank you for help me!
LLogan M
Yea, when you set it to a high value, it will "refine" a response since all the text cannot fit into a single LLM call

Basically, after the first call, it shows the LLM the existing answer and new context, and asks the LLM to update or repeat the existing answer
aaiua
Okay, Thank you!
Add a reply
Sign up and join the conversation on Discord