Hey Folks, I am relatively newbie to RAG

At a glance

Hey Folks, I am relatively newbie to RAG's
[Question]: How to append a newly created KG index to an existing KG index using llama index?
Let's say I created a query engine and asked a question with a single pdf file(File 1) and I asked some questions to the query bot and it answered my questions from the created KG index for that pdf file(File 1). Now I will ask questions by giving another pdf file(File 2) , now it will create a new KG index but what I am looking for is the newly created index(For File 2) shouldn't override the existing KG index of the old pdf document(File 1)(I need to append the new KG index to the old KG index), is this possible with llama index?
As of now my code is overriding the old index when I give a new pdf file to be indexed.

9 comments

LLogan M

Once you create the index, use index.insert(document) to insert new data into the index

PPav1

Hey @Logan M , Can you elaborate on this please about how to insert, here is my below code to create KG index.

LLogan M

Plain Text

from llama_index import StorageContext, load_index_from_storage
storage_context = StorageContext.from_defaults(persist_dir=index_name_knowledge)

kg_index = load_index_from_storage(storage_context, service_context=service_context)

documents = ...
for document in documents:
    kg_index.insert(document)

PPav1

So insert method will create a new KG index for new documents and will insert it into the existing KG index that we have in the storage context.
Am I correct @Logan M ?

LLogan M

insert will append/insert into to the existing graph

PPav1

I see your code is not creating any new KG index. Does insert method will take care of it?
what I am saying is you are not using any code like this.
kg_index_new = KnowledgeGraphIndex.from_documents(
documents,
storage_context=storage_context,
max_triplets_per_chunk=2,
space_name=space_name,
edge_types=edge_types,
rel_prop_names=rel_prop_names,
tags=tags,
include_embeddings=True,
service_context=service_context,
)
please help

LLogan M

No, I'm assuming you are loading an existing graph (which is what I thought you wanted lol)

PPav1

Yes, my use case is I will load the exsiting graph, but at runtime I need to create the KG index for a new document and append/merge that newly created KG index to existing KG index

LLogan M

You cant merge two graphs. You can only load an existing one and then insert new documents into it

Add a reply

Find answers from the community

Hey Folks, I am relatively newbie to RAG