Find answers from the community

Updated 2 years ago

Also another q seems like increasing

At a glance

Also another q: seems like increasing similarity_top_k from 1 to 3 makes the query much much slower (I am using GPTSimpleVectorIndex). I am wondering why that's the case. And how could I mitigate this latency increase

3 comments

LLogan M

try setting response_mode"compact" in the query. https://gpt-index.readthedocs.io/en/latest/guides/usage_pattern.html#setting-response-mode

LLogan M

It's slow because by default, it would be making 3 LLM calls, which is limited by OpenAI's speed

LLogan M

"compact" will send as many nodes as possible, reducing how many api calls are made

Add a reply