Find answers from the community

Updated 3 months ago

Docstore

It's certainly a basic question but i could'nt manage to find the answer : i have a WeaviateVectorStore from which i have a VectorIndex. I need to load the docstore to iterate some search through all docs, but i dont know how to load the docstore.

Plain Text

 vhector_store = WeaviateVectorStore(
            weaviate_client=client, index_name="Ffnuydhnxulwm"
        )
        index = VectorStoreIndex.from_vector_store(
            vhector_store,
        )
        docstore = index.storage_context.docstore

here, docstore seems empty

I am can query the index without any problem

8 comments

WWhiteFang_Jr

You can load the docstore by adding it in storage_context like this.

Plain Text

# ASSUMING YOU ALREADY HAVE A DOCSTORE

# create storage context WITH DOCSTERE
storage_context = StorageContext.from_defaults(docstore=docstore)

# build index
index = VectorStoreIndex.from_vector_store(
            vhector_store, storage_context=storage_context
        )

For more info on Docstore, You can refer here: https://docs.llamaindex.ai/en/stable/core_modules/data_modules/storage/docstores.html#document-stores

LLogan M

By default, the docstore is not used with vector db integrations. You need to enable it (and also store/load it)

This is showing something slightly more involved (refresh) but the saving and loading part is the same

https://discord.com/channels/1059199217496772688/1163880111074971790/1163900056718553169

WWhiteFang_Jr

Okay so, If not for third party docstore like mongo/redis, We will have to create Document object again?

LLogan M

The nodes are stored in the vector store instead of the docstore by default there. It makes storage easy, but you can override and manually enable the docstore to store nodes again as usual

VVH-TLS

thx a lot for your answers. So, from what i understand, i need to store the Docstore "somewhere" in order to load it after. Is there a way to retrieve all the docs or the nodes stored in the WeaviateVectorStore ?

LLogan M

Unsure tbh. Easiest way would be to just set the top k to 10000 lol

VVH-TLS

😄

VVH-TLS

thx for your time, i will adapt my logic. You guys rock !

Add a reply