I m on a MacBook Pro M2 Max and when I

I get that too 😅 (I have the same macbook)

Never bothered to fix it -- i just use llama.cpp instead, compiled with apple metal support

I think it has something to do with installing torch with apple silicon support? Not sure though

Got it.

I'm a rookie, so very rookie question coming up:

any tips/guides on how to integrate llama.cpp with llama index? is it just getting the model locally via llama.cpp and referencing (how?) in llama index?

thanks again for all the effort and work you put into this!

Right! We have a guide here on llama.cpp

https://gpt-index.readthedocs.io/en/stable/examples/llm/llama_2_llama_cpp.html

This will download and run llama2-13B-chat locally 🙂

To save you time, here's the install command for macs