Find answers from the community

Updated 3 months ago

how can I load multiple indices as a retriever?

how can I load multiple indices as a retriever?
k
J
S
6 comments
Please stand by @Jam Mirzakhalov as I search the knowledge base. I normally take about 20 seconds or so. :wave:
----------------------
You can load multiple indices as a retriever by using the QueryFusionRetriever or RecursiveRetriever classes in LlamaIndex.

For QueryFusionRetriever, you can fuse your indexes into a single retriever. This retriever will also generate extra queries related to the original question, and aggregate the results. Here is an example of how to use it:

Plain Text
from llama_index.retrievers import QueryFusionRetriever

retriever = QueryFusionRetriever(
    [index_1.as_retriever(), index_2.as_retriever()],
    similarity_top_k=2,
    num_queries=4,  # set this to 1 to disable query generation
    use_async=True,
    verbose=True,
    # query_gen_prompt="...", # we could override the query generation prompt here
)

Source ()

For RecursiveRetriever, you can define a separate IndexNode corresponding to the vector retriever for each chunk size, put all IndexNodes into a single SummaryIndex, and then define a Recursive Retriever, with the root node being the summary index retriever. Here is an example of how to use it:

```python
from llama_index.tools import RetrieverTool
from llama_index.schema import IndexNode
from llama_index.retrievers import RecursiveRetriever
from llama_index import SummaryIndex

retriever_dict = {}
retriever_nodes = []
for chunk_size, vector_index in zip(chunk_sizes, vector_indices):
nodeid = f"chunk{chunk_size}"
node = IndexNode(
text=(
"Retrieves relevant context from the Llama 2 paper (chunk size"
f" {chunk_size})"
),
index_id=node_id,
)
retriever_nodes.append(node)
retriever_dict[node_id] = vector_index.as_retriever()

summary_index = SummaryIndex(retriever_nodes)
retriever = RecursiveRetriever(
root_id="root",
retriever_dict={"root": summary_index.as_retriever(), retriever_dict},)```Source** ()

Please replace index_1, index_2, chunk_sizes, and vector_indices with your actual indices and chunk sizes.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
@kapa.ai I have already loaded indices where each object is VectorStoreIndex. How do I get the nodes and create a Recursive Retriever
@kapa.ai I have already loaded indices where each object is VectorStoreIndex. How do I get the nodes and create a Recursive Retriever

index_objects = self.initialize_multiple_indexes(indices=indices)
@Jam Mirzakhalov I am stuck at this same spot on how to send the nodes dictionary to node_dict of RecursiveRetriever from an existing VectorStoreIndex. Were you able to figure this out?
Add a reply
Sign up and join the conversation on Discord