Find answers from the community

Updated 3 months ago

Score

More of a LLM question, but when SubqueryQuestionEngine does retrieval, it then passes the 3 nodes it finds along with the score to the LLM to get an answer. Do the scores have an effect on what the LLM decides to do? I'm wondering why the score is passed into the context as well.

5 comments

LLogan M

The score is not sent to the LLM 🤔 curious where you saw this.

For the sub question query engine, the LLM writes sub questions and which query engines to send them to. It runs those queries with their respective query engines, then sends those questions+responses to a top level synthesis step

cchantlong

Sorry I made a mistake when deciphering the logs. It isn't sent. False alarm. 🙏🏻
Btw, for the SubqueryQuestionEngine, with the new update_prompt API, I assume it's possible to update the following?

question generation prompt
response synthesizer (top level synthesis) prompt

LLogan M

Hmm it should be, although I haven't tested it yet 😅 the top level should be exposed, but you might have to set the prompts for each sub-index before passing them into the sub question engine

cchantlong

What do you mean by sub-index? You mean each QueryTool's prompt?
QueryEngineTool(
query_engine=XXX_vector_query_engine_index.as_query_engine(
similarity_top_k=3),
metadata=ToolMetadata(
name="XX", description="XXX"
),
)

LLogan M

Right, index.as_query_engine(...) -- that's the sub index / sub query engine

Add a reply