Find answers from the community

Updated 8 months ago

llama index is spamming too much of

memory issue
W
T
L
19 comments
For LLM/embed/index ?
You can use them from remote if they are causing you memory issue.

Or is it something else?
it cause memory load
after awhile it doesnt clear
these node take alot of memory in gpu and overexceed from 20gb from 8gb
sounds like you are using a local model? They allocate up to a point. I've never had memory issues, but you need to configure some settings properly (batch size, llm context window, using an actual vector store if you have a lot of data, etc.)
the problem with this after 8gb it exceed 20 then it fallback to cpu
1 single call expand almost 2-4gb vram from llama index
while ollama pure call not expand that much also reset back to 8gb
llama index also make memory clean but i dont know it very slow like take almost 2-3min if ideal state
1 person cause this much rise of vram how can it survive
also notice it wont clean the memory anymore
Attachment
image.png
it stuck at this point unless wait for more then 1-2min i am not sure about time but it do clean after bit idle state if keep spam it fallback to cpu and hence it go very slow
@Logan M i found it solve by the new ollama3 experimental update
it seems like the ollama itself has problem in earlier version
now it fixed i am happy now thnx alot bro >.</ for coming
Add a reply
Sign up and join the conversation on Discord