Find answers from the community

Updated 4 months ago

Hi guys, I'm getting mad... Is there a

Hi guys, I'm getting mad... Is there a way to update a created vector store index adding documents with embeddings and all?
L
P
6 comments
index.insert(document) ?
Or do you mean updating existing documents
You kind of need to have kept track beforehand while inserting to updating existing. You have to insert a document witht the same document ids as the original, usually with an ingestion pipeline

https://docs.llamaindex.ai/en/stable/module_guides/loading/ingestion_pipeline/#document-management
I now have this:

chroma_collection = db.get_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

vector_store_index = VectorStoreIndex.from_vector_store(
vector_store,
embed_model=embed_model,
)
print("Index loaded from storage.")

new_documents = load_new_documents(folder_path)
if new_documents:
# Parse documents into nodes
print("Parsing new documents into nodes...")
parser = SimpleNodeParser()
new_nodes = parser.get_nodes_from_documents(new_documents, show_progress=True)


# Add nodes to the existing index
print("Adding new nodes to the existing index...")
vector_store_index.insert_nodes(new_nodes)

print("New documents added to the index.")
the fact is that in this way the embedder model never gets called, the nodes are added as "plain text" not as embeddings
How can the emded model not get called?

insert_nodes() has to call the embed model πŸ˜…
Add a reply
Sign up and join the conversation on Discord