hi all I m trying to figure out how to

At a glance

The community member is trying to reload an index from persistent storage using Faiss, but is encountering errors. The community members discuss the challenges of using Faiss for persistence, and suggest alternative solutions like Qdrant or Weaviate that may provide better persistence capabilities. One community member provides a potential solution by specifying the persist_dir when storing and loading the index, which seems to have worked for the original poster.

vvcb

hi all, I'm trying to figure out how to reload an index from persistent storage but I'm getting errors:

Plain Text

#create an index
d = 1536
faiss_index = faiss.IndexFlatL2(d)
vector_store = FaissVectorStore(faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = VectorStoreIndex.from_documents(storage_context=storage_context,documents=documents)
#index works

#to store:
index.storage_context.persist()

#reload
vector_store = FaissVectorStore.from_persist_dir('./storage')
storage_context = StorageContext.from_defaults(vector_store=vector_store)
#this index doesnt work
index = load_index_from_storage(storage_context=storage_context)
#ValueError: No index in storage context, check if you specified the right persist_dir.

7 comments

LLogan M

oof faiss. Not 100% sure how persistance works with that

LLogan M

oh I see maybe

LLogan M

Plain Text

#to store:
index.storage_context.persist(persist_dir="./storage")

#reload
vector_store = FaissVectorStore.from_persist_dir(persist_dir='./storage')
storage_context = StorageContext.from_defaults(vector_store=vector_store, persist_dir="./storage")
index = load_index_from_storage(storage_context)

LLogan M

does that help maybe? I see where you got the example from, but the notebook seems a little wonky. Faiss is not used very often 😅

vvcb

Oh interesting. It was my assumption that faiss is the fastest in-memory store

vvcb

That worked by the way thanks man!

LLogan M

amazing! I will update that notebook lol

Yea it's fast, but most people want something with better persistance (i.e. qdrant, weaviate, etc.). For what it's worth, qdrant is also super fast

Add a reply

Find answers from the community

hi all I m trying to figure out how to