My Requirement is : When updating RAG,

At a glance

The community member's requirement is to generate embeddings only for newly added documents when updating RAG, rather than regenerating embeddings for all documents in the Chroma DB using LlamaIndex. In the comments, another community member suggests using the insert method to add new documents, which will only create embeddings for the newly inserted docs. Additionally, the community members discuss how to remove a document from the index using the delete_ref_doc(ref_doc_id) method, which will delete all nodes associated with that reference document ID.

AAmitKhandey

My Requirement is : When updating RAG, only generate embeddings for newly added documents rather than regenerating embeddings for all documents in chroma db using llamaindex.

4 comments

WWhiteFang_Jr

How are you currently adding new docs?
You can insert new docs using insert method that will only create embedding for the newly inserted docs.

Sample code would look like

Plain Text

docs = newly created documents

for doc in docs:
  index.insert(doc)

AAmitKhandey

Can I remove a doc ?

LLogan M

yes -- index.delete_ref_doc(ref_doc_id)

LLogan M

will delete all nodes associated with that ref doc id

Add a reply

Find answers from the community

My Requirement is : When updating RAG,