Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
๐
๐
๐
Powered by
Hall
Inactive
Updated 2 months ago
0
Follow
i'm seeing that when i use the
i'm seeing that when i use the
Inactive
0
Follow
s
shawtyisaten
11 months ago
ยท
i'm seeing that when i use the ReactAgent with vector db, it can take up to 1 minute for it to come up with an answer for a complex query. Is there anything we can do to boost the speed?
L
s
6 comments
Share
Open in Discord
L
Logan M
11 months ago
Are you using a local LLM?
s
shawtyisaten
11 months ago
i'm using OpenAI's GPT
L
Logan M
11 months ago
And it takes 1 minute to answer? I've really never seen that before. Is it calling the query engine tool a ton of times?
s
shawtyisaten
11 months ago
it calls the query engine tool maybe up to 3 times.
s
shawtyisaten
11 months ago
but the context string is pretty long (but max 512 tokens) can that be why?
L
Logan M
11 months ago
Calling 3 times is probably the main reason -- thats quite a few LLM calls to make, each LLM call takes 3-10s usually.
Add a reply
Sign up and join the conversation on Discord
Join on Discord