Find answers from the community

Updated last year

hey im trying to follow the quick start

At a glance

hey im trying to follow the quick start, but with ollama, however i get this error

Plain Text

llama.cpp: loading model from /Users/my_user/Library/Caches/llama_index/models/llama-2-13b-chat.ggmlv3.q4_0.bin
error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file?
llama_init_from_file: failed to load model

13 comments

LLogan M

I think you'll need to pass the service context into the index instead

index = VectorStoreIndex.from_documents(documents, service_context=service_context)

IIceCap

interesting, it works.

IIceCap

is this a discrepancy in the docs ?

IIceCap

Attachment

LLogan M

I think it's just that the initial from_documents() is creating a default service context because you didn't pass one in. And since you don't have an openai key set, it's defaulting to llama.cpp, and it seems like something there is not quite working 😅

IIceCap

interesting

IIceCap

much to learn

IIceCap

but yeah so technically, the snippet in the screenshot above would bug in the same way right

IIceCap

since there's no openai key, and no service context when calling from_documents()

LLogan M

tbh, most examples in the docs assume you have the openai key set lol

But even if you didn't, I think something is just wrong with your llama-cpp install/cached model file

IIceCap

i see, so the openai key configures more than just the llm ?

LLogan M

It determines some of the default settings yes.

A lot of users in the past were frustrated that openai was being used even though they explicitly didn't set their openai key, so we added this sort of fallback to llama.cpp

Add a reply