LlamaIndex
Log in

Find answers from the community

Updated 2 months ago

I ran into cuda OOM error when running

vvictor
·
I ran into cuda OOM error when running Gemma 7b on ollama. My GPU has 8G memory and can run llama2 7b and Mistral 7b without issues. I could run Gemma 7b in ollama CLI though, just not via llamaindex in a rag app.
W
v
2 comments
WWhiteFang_Jr
With Ollama in LlamaIndex, you interact with your hosted LLM.

Were you facing issue while interacting via llamaIndex?
vvictor
yeah. I was building a simple txt file rag app. when i run gemma 7b it ran into the cuda out of memory error.
Add a reply
Sign up and join the conversation on Discord