Find answers from the community

Updated 12 months ago

How can I load a persisted index when I'

How can I load a persisted index when I'm using a local LLM?
I am using this code:

storage_context = StorageContext.from_defaults(persist_dir="../data/datastore") index = load_index_from_storage(storage_context, llm=llm)
where the llm is an instance of LLamaCPP.
I get this error: Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
W
r
L
20 comments
you'll need to pass in the service_context in load_index_from_storage too
remove the llm part
load_index_from does not accept a ServiceContext.
it does

Plain Text
vector_index = load_index_from_storage(storage_context, service_context)
index = load_index_from_storage(storage_context, rag_service_context)
TypeError: unhashable type: 'ServiceContext'
it will make the newly loaded index to use the passed service-context for llm operation
What version are you trying with?
Try passing it this way

load_index_from_storage(storage_context, service_context=service_context)
Tried it already 😦
Same error (unhashable type)
let me check on this
Thanks a lot for your speedy help - really appreciated.
worked for with 0.9.7, Can you share you code so that I can take a look
The vector store is created in another notebook; I can upload that if it would help.
That's the notebook that is failing, not the one that creates and persists the VectorStoreIndex.
Plain Text
from llama_index import Document, ServiceContext, VectorStoreIndex, Document, load_index_from_storage, StorageContext


# define your service context

service_context = ServiceContext.from_defaults(
    llm=llm,embed_model="local:BAAI/bge-small-en-v1.5"
)

# load data
pg_essay = [Document(text="this is first data")]

# build index and query engine
vector_index = VectorStoreIndex.from_documents(
    pg_essay, use_async=True, service_context=service_context
)
print(vector_index.service_context.embed_model)

vector_index.storage_context.persist()
storage_context = StorageContext.from_defaults()
# load a single index
# need to specify index_id if multiple indexes are persisted to the same directory
index = load_index_from_storage(storage_context, service_context=service_context)

print(index.service_context.embed_model)


replace the llm with your llm and check once running in a fresh colab session
also do this at the top
Plain Text
!pip install llama_index==0.9.7
!pip install transformers
Thanks. I now have a tweaked version of your code running locally, and can see how to fix my code.
I think it's probably best to specify it as a kwarg

vector_index = load_index_from_storage(storage_context, service_context=service_context)
Add a reply
Sign up and join the conversation on Discord