Find answers from the community

Updated 4 months ago

i'm seeing that when i use the

At a glance

i'm seeing that when i use the ReactAgent with vector db, it can take up to 1 minute for it to come up with an answer for a complex query. Is there anything we can do to boost the speed?

6 comments

LLogan M

Are you using a local LLM?

sshawtyisaten

i'm using OpenAI's GPT

LLogan M

And it takes 1 minute to answer? I've really never seen that before. Is it calling the query engine tool a ton of times?

sshawtyisaten

it calls the query engine tool maybe up to 3 times.

sshawtyisaten

but the context string is pretty long (but max 512 tokens) can that be why?

LLogan M

Calling 3 times is probably the main reason -- thats quite a few LLM calls to make, each LLM call takes 3-10s usually.

Add a reply