Hi, when I use storage

At a glance

The community member is having an issue with the index.docstore.docs.values() being empty when using storage_context with Supabase. The issue does not occur when they try without storage_context. Another community member explains that the docstore is disabled when using a vector database integration to simplify storage, and suggests overriding it if needed, but the user would need to handle storing it somewhere. The second community member provides a solution involving persisting the docstore/index store locally and loading it later.

MMrMJ

Hi, when I use storage_context with supabase, I can't get the 'index.docstore.docs.values()'. It's empty. It works when I tried without storage_context, it outputs the nodes.

Plain Text

node_parser = SimpleNodeParser.from_defaults(chunk_size=250)
embed_model = OpenAIEmbedding()
vector_store = SupabaseVectorStore(postgres_connection_string=DB_CONNECTION, collection_name='index_main', dimension=1536)
service_context = ServiceContext.from_defaults(embed_model=embed_model, node_parser=node_parser)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
documents = Document(text="I like to eat apples. ")

index = VectorStoreIndex.from_documents([documents], service_context=service_context, storage_context=storage_context)

print(index.docstore.docs.values())
# output => dict_values([])

3 comments

LLogan M

the docstore is disabled when using a vectordb integration, in order to simplify storage

If you need it, you can override it, but you'll need to handle storing it somewhere (either to disk, or using a docstore integration like redis or mongodb)

LLogan M

Plain Text

storage_context = StorageContext.from_defaults(vector_store=vector_store)

# create index
index = VectorStoreIndex.from_documents(
  documents, 
  storage_context=storage_context, 
  service_context=service_context, 
  store_nodes_override=True
)

# save the docstore/index store locally
index.storage_context.persist(persist_dir="./storage")

# load the index
new_storage_context = StorageContext.from_defaults(vector_store=vector_store, persist_dir="./storage")

from llama_index import load_index_from_storage
# optional service context
loaded_index = load_index_from_storage(new_storage_context, service_context=service_context)

MMrMJ

I see. Didn't expect the answer in this late time. Thanks a lot! 👍

Add a reply

Find answers from the community

Hi, when I use storage_context with