Graph

Sorry, just want to understand the structure you have a bit more.

You have 42 documents, and you are using a graph to pick which documents to query? Or something else?

I have 42 documents for EACH separate document store.

Based on the user question:

1) one or more of the documents stores will be selected by the relevance to the question
2) once those stores are selected , the summary nodes of the graph are then queried within the store(s) to retrieve the relevant nodes.

Previously

#1) I was doing with concatenating 3 query engine tools and wrapping it around a SubQuestionQuery Engine and no impact to current process

#2) I was doing with Composable graph with retriever_mode = LLM which would cause the composable graph to a) query the summary index of each document (rather than the full text of each node) then b) select the relevant documents after choosing the relevant documents. This is the impacted part of my process.

Problem

I was told that Composable graph was ditched for other developments but I do not now how to get the same functionality of querying summaries of document within a graph THEN selecting the relevant documents

@Logan M

I feel like you can achieve the same with a router

https://docs.llamaindex.ai/en/stable/module_guides/querying/router/root.html#routers

Thanks, what would you recommend for the query engine? in the example I gave I had 42 documents in each graph then used graph as query engine, I understand the LLMMultiSelector may be a good option. Before in the graph I would have each document be a node (42 nodes) where the retrieval (after filtering by querying SummaryINdex) would select the full document node.

So 42 nodes is a lot for the LLM to make a good decision based off of.

You might have better luck using embedding router?
https://docs.llamaindex.ai/en/stable/examples/query_engine/RetrieverRouterQueryEngine.html

Or you can manually do some embedding step to narrow down the choices before doing the llm selector.

The chosen indexes would then be queried and all the responses aggregated for a final response

The graph query would allow me to query all 42 summary documents, then just select one. it would be 42 small queries ( 20 tokens each) then retrieve the full document (10k + tokens). I worry embedding would cause issues on the question. The question will say "in Q1, in Q2 etc how did NNN do? " and the document summary says "Q2 summary for NNN". It has to be exact in nature when it retrieves, dont want other NNN chosen. The retriever_mode = 'llm" was super useful as it was a discretionary query to each of the summary indexes rather than an embedding search.

Thats fair, then you can use the LLM selector if that was working fine before with that number of documents

I was using composable graph, with each node document being a node within the graph. I guess I am just not sure how to compile an index that would allow me to store 42 documents, including their summaries, and would allow me to easily query.

Sorry if I am confused and not understanding your suggestion yet, appreciate your guidance greatly.

Thinking maybe:

Summary index with each node being a document, then someway to route the query (after tool selection with LLMMultiSelector) to the summary within a summary index before the final node is selected, am I making sense?