Find answers from the community

Updated 10 months ago

Can I merge multiple vector store index

Can I merge multiple vector store index that built previously?
W
R
5 comments
Yeah you can. Just take the nodes from other indexes and add it to the one you want to keep.


Plain Text
nodes1 = index1.docstore.docs # Extract nodes from the existing index

for node in nodes1:
     final_index.insert(node)

## ONce all the nodes are added, persist the index.

PS: This will only work if you have the nodes in your docstore
Thanks!! But the nodes are empty by using this method, and I checked the chromadb by retrieving, they are not empty. Here is my code:
Plain Text
# load db
db = chromadb.PersistentClient(path=db_path1)
chroma_collection = db.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
index = VectorStoreIndex.from_vector_store(vector_store, service_context=ServiceContextModel()())

# load db_final
db_final = chromadb.PersistentClient(path=db_path2)
chroma_collection_final = db_final.get_or_create_collection("quickstart")
vector_store_final = ChromaVectorStore(chroma_collection=chroma_collection_final)
index_final = VectorStoreIndex.from_vector_store(vector_store_final, service_context=ServiceContextModel()())

nodes = index.docstore.docs
for node in nodes:
    index_final.insert(node)
There was a arg called store_node_override or something. That would fetch all the nodes back to docstore. I may be wrong
But since you have the chroma client with you, fethc all the stored nodes from the DB using client and then you can extract the text and metadata, create a Document and insert it into your final index
OK, I got it, thank you☺️
Add a reply
Sign up and join the conversation on Discord