Hello everyone,
I intend to utilize openchat_3.5 as my Language Model (LLM) instead of ChatGPT for Retrieval-Augmented Generation. To achieve this, I've successfully downloaded the openchat_3.5.Q8_0.gguf model onto my computer. I'm employing the llama_cpp library to establish a connection to the LLM, as illustrated below:
from llama_cpp import Llama
llm = Llama(model_path="/Users/developer/ai/models/openchat_3.5.Q8_0.gguf", n_gpu_layers=1, n_ctx=2048)
Now, I'm seeking guidance on how to link LlamaIndex to the local LLM, such as openchat_3.5.Q8_0.gguf.
Thank you.