Find answers from the community

Updated 2 months ago

I have a .index file(faiss) and i want

I have a .index file(faiss) and i want to load it but i get this error
Attachment
image.png
W
A
27 comments
Hi, did you follow this https://docs.llamaindex.ai/en/stable/examples/vector_stores/FaissIndexDemo/?h=faiss
doc ?

If not I would suggest you go through this once
I did not, i transferred the nodes via storage context manually cause they were a lot
How did you created the embeddings?
Initalized a Vector Store Index on documents , they were stored in that
After words i persisted those with index.storage_context.persist()
it works but it's too slow so i thought of shifting to faiss, the documentation seemed to complex so now i am at chromadb
The only problem i have now is that I don't know how to add a new nodes in the same index and i do not know if they get persisted to chroma as well
All your previous nodes have been added to chroma?

If all the previous ones are added to chroma then you dont have to use Faiss at all
Yes I dumped faiss
simply connect with Chroma and if you do index.insert(doc) it will insert new doc into Chroma
is this a list of documents?
like the one simpledirectoryreader creates?
Yes but you'll have to iterate and insert one item at a time
is there batch option?
Plain Text
docs = SimpleDirectoryReader(...)
for doc in docs:
  index.insert(doc)
And it will automatically create and persist on chroma?
i just have to pass the storage_context parameter right?
To create the index right ? YES
Plain Text
storage_context = StorageContext(vector_store=chroma_vector_store_ref)
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)
shpuldn't it be this
VectorStoreIndex.from_vector_store(vector_store=vector_store, stroage_context=storage_context)
In case if you do not need docstore built earlier or anything else apart from vector_store then it will work without passing storage context as well.
It just needs to be the same collection and same persistant path right while loading the index
Can i use insert_nodes? will those create the embeddings?
Yes that will also create embeddings
Add a reply
Sign up and join the conversation on Discord