LlamaIndex
Log in

Find answers from the community

Updated 2 months ago

i'm seeing that when i use the

sshawtyisaten
·
i'm seeing that when i use the ReactAgent with vector db, it can take up to 1 minute for it to come up with an answer for a complex query. Is there anything we can do to boost the speed?
L
s
6 comments
LLogan M
Are you using a local LLM?
sshawtyisaten
i'm using OpenAI's GPT
LLogan M
And it takes 1 minute to answer? I've really never seen that before. Is it calling the query engine tool a ton of times?
sshawtyisaten
it calls the query engine tool maybe up to 3 times.
sshawtyisaten
but the context string is pretty long (but max 512 tokens) can that be why?
LLogan M
Calling 3 times is probably the main reason -- thats quite a few LLM calls to make, each LLM call takes 3-10s usually.
Add a reply
Sign up and join the conversation on Discord