Find answers from the community

Updated 3 months ago

can i use llama-cpp-python with

can i use llama-cpp-python with llamaindex to run llama3 fine tune model in gguf format for hugging face?

3 comments

huggingface just added support for loading gguf files, so yes

Using HuggingFaceLLM

just load the model and tokenizer, and pass it in mostly

HuggingFaceLLM(model=model, tokenizer=tokenizer, ...)

Add a reply