Hey there been working with the

JJjonus

Hey there, been working with the subquestion query engine example comparing uber and lyft financial docs and running into some issues (and spammed kapa bot on it):

the index_1 and index_2 are base TreeIndex that I am able to query against individually. However when I run the code below I get the error at the bottom. Any clue?

I am attempting to build sub questions on two root tree indices and then refine the response in the end, similar to the uber and lyft example but just using tree index.

Plain Text

 from llama_index.tools import QueryEngineTool, ToolMetadata
from llama_index.query_engine import SubQuestionQueryEngine

index_1 = indices_list[0].as_query_engine(retriever_mode = "root",use_async=True)
index_2 = indices_list[1].as_query_engine(retriever_mode = "root",use_async=True)

query_engine_tools = [
    QueryEngineTool(
        query_engine=index_1,
        metadata=ToolMetadata(
            name=list(loaded_indices.keys())[1].strip(".json"),
            description=f'This is the earning transcripts for {list(loaded_indices.keys())[1].strip(".json")}',
        ),
    ),
    QueryEngineTool(
        query_engine=index_2,
        metadata=ToolMetadata(
            name=list(loaded_indices.keys())[2].strip(".json"),
            description=f'This is the earning transcripts for {list(loaded_indices.keys())[2].strip(".json")}',
        ),
    ),
]
s_engine = SubQuestionQueryEngine.from_defaults(
    query_engine_tools=query_engine_tools,
    service_context=service_context
)

response = s_engine.query("Please compare the performance of these two companies")

Error:

/opt/conda/lib/python3.8/site-packages/llama_index/question_gen/output_parser.py in format(failed resolving arguments)
15 def format(self, prompt_template: str) -> str:
16 del prompt_template
---> 17 raise NotImplementedError()

NotImplementedError:

P.S I am on llama_index version 0.8.30

14 comments

LLogan M

Update to the latest llama-index version, this should be fixed

JJjonus

I got it to work now, thank you so much! I had a quick follow up question. My intuition with the subqueries is that:

The subquestion will be asked against each node in each index refining one index at a time (in sequence from the root nodes 1-N) since I am calling the root node.
The original query (to compare) will take the two indices of refined subquestions and synthesize a response.

Is that assumption correct ? Wasnt quite sure. Thanks again for all the help!

LLogan M

That's nearly how it works!

Your original query is decomposed into sub-queries. Each sub query is directed to a specific index that the LLM thinks was best

Then the responses are used for one last synthesis step using the original query

JJjonus

Ah ok, thanks so much! So if I were to want to mix nodes on the sub-questions, it doesnt seem to be doing that. I was hoping it would ask the sub questions of each node then synthesize from there.

LLogan M

Not quite -- it reads the description of each sub index, and figures out which query to send where

LLogan M

It could send the same question to multiple if it's ambiguous

JJjonus

But description is at the index level not at node level. What if I am calling 6 summary root nodes by retriever for each of the 2 index . Does it send the single sub question to one of the root nodes with the index it chose to build a pair against or match against node(s) within the chosen index ? From your answer it almost seems like it refines the sub question within the chosen index (from all the root nodes).

LLogan M

right. I don't know what this means: What if I am calling 6 summary root nodes by retriever for each of the 2 index , but maybe I can explain more clearly

The sub question query engine takes a list of query engine tools.

These tool can be any query engine -- a vector index query engine, another sub-question query engine, anything.

Each tool has a description

The LLM reads the user message + all the tool names/descriptions

It writes one or more queries to send to a specific sub index. There's no constraints on this, it's up to the LLM to decide

It then sends each sub-query to each sub index. It runs the full query engine for each sub index (i.e. retrieveal + synthesis)

Then, it takes all the responses from each sub-index, and sends them to the LLM, along with the original user query, to generate a final response.

LLogan M

Here's a lovely drawing 😆

Attachment

JJjonus

ah ok, so in this case, if I am using SelectTreeRootRetriever in this diagram in the retriever part of the query engine, it should then select all the root nodes to produce the response?

JJjonus

in each of the divergent paths

LLogan M

Exactly -- it runs the full query engine. So if you have a query engine tool that uses a tree retriever, it will use that, and then produce a sub-response for that tool

JJjonus

awesome, so it is doing as I hoped. refine synthesis from the root nodes in each of those paths

JJjonus

Need to tweak the summary though through refine, it seems quite strict in the information it passes on

Add a reply

Find answers from the community

Hey there been working with the