Find answers from the community

Updated 4 months ago

Gpu

At a glance

it looks like it's been asked previously but no clear answer and the docs aren't super clear either -- how do I make the embedding/indexing use GPU? or, how do I know if it used/might use GPU?

6 comments

LLogan M

You can set device="cuda" in the constructor, but it should automatically use the gpu if cuda is available

You can check nvidia-smi in your terminal to view if a python process is using gpu memory

tthoraxe

i thought that was the case

tthoraxe

waiting for this environment to restart. i was getting oomkilled because it hink it wasn't picking up the gpu

LLogan M

Hmm interesting

I know you linked a page on onnx/Optimum embeddings earlier. Using GPU for those is a little more tricky (tbh I never got it to work)

tthoraxe

yeah no not doing the optimum stuff

tthoraxe

just the regular stuff

Add a reply