Hi everyone,

At a glance

Hi everyone,
I am building a RAG app where I want the user to be able to select the files she wants to query (using the streamlit "multiselect" widget).
I am unsure what is the best way to implement that. My first idea was to create an index for each file, and then load and merge the indices as necessary. But it seems Llamaindex is not great at merging indices? I have seen the solution 'index = GPTListIndex([index1, index2])' is mentioned multiple times in the Github help section but it’s not working for me (it should receive nodes as inputs and not indices if I’m not mistaken). I have also seen the possibility to use the index.insert() and delete_ref_doc() function, but I’m not sure how to keep track of all the ids of my documents.

I’m also open to other solutions using only one index (and filtering nodes with a "tag" metadata, using graphs...)

4 comments

WWhiteFang_Jr

You can keep all the indexes in a dict like

Plain Text

{
query_engine_A: index_A.as_query_engine(),
query_engine_B: index_B.as_query_engine()
}

based on the file selected you can pick the engine and perform the ops

Second way I can think of is to use Router query engine and define instructions to each engine and let user query and instruction choose the engine.

More here: https://docs.llamaindex.ai/en/stable/examples/query_engine/RouterQueryEngine.html

SStéphane

@WhiteFang_Jr thanks. Second solution is not really adapted to my case as some of my files are fairly general and cannot be described easily. A human will do better than the code at selecting the right index.

For the first solution you suggest, I forgot to mention that sometimes multiple files will be selected at the same time, it’s not necessarily one or the other. Hence my question about the best way to merge indexes in this case.

LLogan M

Yea merging is not really possible.

You can load an existing index and append to it using index.insert(document)

LLogan M

Could also use this, which queries all indexes at the same time and returns the top k from all indexes

https://docs.llamaindex.ai/en/stable/examples/retrievers/simple_fusion.html

Add a reply

Find answers from the community

Hi everyone,