Find answers from the community

Updated 2 months ago

Hello I have a newbie question on

Hello I have a newbie question on working with Pinecone:

So I do:

index = GPTVectorStoreIndex.from_documents(
documents, storage_context=storage_context,
service_context=service_context
)

My question is, if my documents get updated and I run the above code again, does my entire index gets replaced by the new document embeddings, or the new embedding simply gets added to the existing index?

5 comments

ddisiok

currently the default behavior is upsert. so here the updated documents will be split again, generating new nodes, and those will get added to the same collection in pinecone.

TTony L

Thanks Simon, so how would querying behave if I have duplicate nodes?

ddisiok

it would definitely be suboptimal, since the retriever would most likely pickup multiple versions of the same doc.

We do have a refresh endpoint as part of our indices, which you can use to refresh an index with documents that have changed.

Take a look at: https://docs.llamaindex.ai/en/stable/core_modules/data_modules/index/document_management.html Might be helpful for you!

TTony L

Thank you! this is helpful

ddisiok

no problem!

Add a reply