Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
π
π
π
Powered by
Hall
Inactive
Updated 8 months ago
0
Follow
why does it take 15-20 seconds to get an
why does it take 15-20 seconds to get an
Inactive
0
Follow
m
matter of factly
8 months ago
Β·
why does it take 15-20 seconds to get an answer when I use ollama(llama3) with llama_index?
L
m
4 comments
Share
Open in Discord
L
Logan M
8 months ago
becuase running a local LLM (especially when using the full context window) takes a lot of resources and can be slow depending on your hardware and what libraries you use to run the model
m
matter of factly
8 months ago
I expected that something that takes no time in the terminal should not take any time in the .py file
L
Logan M
8 months ago
The bigger the input, the longer it takes
L
Logan M
8 months ago
I doubt in the terminal you are pasting I'm several paragraphs as an input
Add a reply
Sign up and join the conversation on Discord
Join on Discord