LlamaIndex
Log in

Find answers from the community

Updated 3 days ago

ollama

oomarhelwe
·
when i use query engine with Ollama llm locally , i get these sometimes:
idk why is it exceeding context window even though im using top_k=1 only and a small prompt
W
o
3 comments
WWhiteFang_Jr
Hi,
  • is your ollama llm running ?
  • Are you running llm on a CPU machine ? you can try increasing the timout for your llm instance.
Settings.llm = Ollama(...,request_timeout=600.0)

default timeout is 30.0 sec
oomarhelwe
yes on cpu , im using timeout 120
WWhiteFang_Jr
I think you need to increase the timeout even further, CPU takes a lot of time
Add a reply
Sign up and join the conversation on Discord