LlamaIndex
Log in

Find answers from the community

Home
Members
puppy333
p
puppy333
Offline, last seen 3 months ago
Joined September 25, 2024
ppuppy333
·

Yea

I have a milvus vector database and mongodb database for docstore and indexstore. I have created an index with VectorStoreIndex.from_documents using the following StorageContext:

Plain Text
storage_context = StorageContext.from_defaults(vector_store=vector_store,
                                                   docstore=docstore,
                                                   index_store=index_store)

    index = VectorStoreIndex.from_documents(docs, storage_context=storage_context, service_context=service_context, store_nodes_override=True)


So atm I have a collection in milvus and two databases and some collections in mongodb.

Is there a specific way to recreate this index later using the docstore and indexstore from mongodb and the specific collection from milvus?

I've been trying for awhile now to recreate the index exactly as I have it here, but nothing seems to give back the doc informaton in the docstore.

The closest I've been able to get is creating an index with the mongodb storage context but no vectorstore, or a vectorstore but without the storage context from mongodb.

Edit: The main issue seems to be that the source_nodes information is always pulled from the vector database even though that is not where the information is actually mostly stored which is in the docstore
4 comments
p
L
ppuppy333
·

I have set up a basic test to see how

I have set up a basic test to see how the vector, doc, and index data is separated and stored. I see when creating an index with a basic doc I get useful information in the docstore.json file like doc_hash, ref_doc_id, etc. under "docstore/metadata", and I get node info under "docstore/data" like extra_info, node_info, etc. But when I change over to use a milvus vector store none of the node data under "docstore/data" is present anymore. That data doesn't show up in milvus either. Is this how combining those two storage methods is intended to work?
3 comments
p
L