llama index is spamming too much of

For LLM/embed/index ?
You can use them from remote if they are causing you memory issue.

Or is it something else?

it cause memory load

after awhile it doesnt clear

these node take alot of memory in gpu and overexceed from 20gb from 8gb

sounds like you are using a local model? They allocate up to a point. I've never had memory issues, but you need to configure some settings properly (batch size, llm context window, using an actual vector store if you have a lot of data, etc.)

sorry back

the problem with this after 8gb it exceed 20 then it fallback to cpu

1 single call expand almost 2-4gb vram from llama index

while ollama pure call not expand that much also reset back to 8gb

llama index also make memory clean but i dont know it very slow like take almost 2-3min if ideal state

@Logan M

1 person cause this much rise of vram how can it survive

also notice it wont clean the memory anymore

it stuck at this point unless wait for more then 1-2min i am not sure about time but it do clean after bit idle state if keep spam it fallback to cpu and hence it go very slow

@Logan M i found it solve by the new ollama3 experimental update

it seems like the ollama itself has problem in earlier version

now it fixed i am happy now thnx alot bro >.</ for coming

great!