Question I have composable graphs that

JJjonus

Question, I have composable graphs that contain data on a certain entities , over time. So rather than having one index for 5 entities for 4 time frames (20 tools) , I can have 5 composable graphs with 4 documents (for each time frame). However, when running the composable graph, I notice it doesnt actually route the query - it just queries ALL of the documents, then chooses the most relevant selection. This causes costs to balloon. Having the tools grow over time is not scalable.

Interested to hear your thoughts on who I could make this work better? in theory I was expecting the composable graph to ROUTE the query rather than querying everything.

7 comments

aandrei

hey @Jjonus!

just so that I understand you correctly, do you mean you have something like you have a sequence of nodes that you want to run in succession; but instead of execuitng one by one, they are all executing at each step?

JJjonus

@nerdai For example, I have a composable graph with 20 articles where there is a summary index and each document is one super large node ( to pull all into context). When querying the composable graph, query goes to each index, all 20. What I would like, is some filtering or routing of the query in the composable graph to the correct summary index. This would reduce costs heavily, however I do not see this functionality. The reason why I am going composable graph route is because if I have to create query engine tools with 20 tools (will be 40 tools by end of year) that it would create problems with the amount of tools to select on each query. If , however, I have 3-4 composable graphs with 20 documents each, I can 1) first route the query to the right tool (composable graph) and 2) within the composable graph route to the right. summary index.

aandrei

thanks @Jjonus for the clarification. I believe now I get what you're saying. I'm diving in on this now.

aandrei

Okay, thanks for your patience. So it seems to me that you're using SummaryIndex as the root BaseIndex type. Perhaps building it using pseudo-code like

Plain Text

graph = ComposableGraph.from_indices(
    SummaryIndex,
    [index1, ..., index20]
    index_summaries=[idx1_summary, ..., idx2_summary]
    storage_context=storage_context
)

If this is true, then for the root SummaryIndex, the default retriever mode, it will use SummaryIndexRetriever that just retrieves all of the nodes -- in this case all of the 20 index. Based on what you're describing, I feel like you're using default retriever mode.

You can do some "routing" based upon the summary texts. Depends on what kind of logic you would like to use. If you change retriever mode to retriever_mode=embedding then it will perform a top-k similarity between the query and the summary_index and use only the top-k retrieved documents/nodes to answer the query.

aandrei

there is also retriever_mode=llm for SummaryIndex that will use an LLM to decide which node/document is best to go to given the user query.

aandrei

Alternatively, you can look to build use a different root BaseNode type other than SummaryIndex. For example: KeywordTableIndex. https://llamaindex-llama-index.readthedocs-hosted.com/en/seldo-docs_reorg_v0/module_guides/indexing/index_guide.html#keyword-table-index

JJjonus

thanks much @nerdai this helps alot

Add a reply

Find answers from the community

Question I have composable graphs that