LlamaCPP

I have no idea what the interface is, but if it's similar to openai API requests, you can use this (I guess there's no example yet, rip)


https://github.com/run-llama/llama_index/blob/95e107423664812eeece1af0f162c9dcd4bfe670/llama_index/llms/openai_like.py#L9


Or you'll have to implement a custom LLM and manually send the requests
https://gpt-index.readthedocs.io/en/stable/module_guides/models/llms/usage_custom.html#example-using-a-custom-llm-model-advanced

im trying to connect to my llama.cpp api I have running locally on a different machine. I compiled the llama.cpp source to run with clblas running 6 gpus. my llama.cpp instance is serving a llama 2 based model.

do I just replace the "llm = LlamaCPP" with the openai_like code?

I thiiiink so?