Find answers from the community

Updated last month

Issues in router query engine

At a glance
I am facing issues in RouterQueryEngine, I have around 6 tools that I am passing to it, and I get some werid behaviour in which it provides right explanation of the tool that it want to select but it returns wrong index number. Due to this the RouterQueryEngine ends up choosing wrong QueryEngine to execute.

This doesn't happen reqularly but its quite frequent though. Any tips to solve this would be greatly appreciated
L
d
10 comments
seems like the llm is having trouble counting in the selector πŸ€” What LLM are you using?
yes its having hard time choosing the right index, i am using GPT-4 0613 version
ooc, how are you setting up the router?
Plain Text
   
query_engine = RouterQueryEngine.from_defaults(
        query_engine_tools=[
            sql_tool,  # index=0
            vector_tool,  # index=1
            summary_tool,  # index=2
            fetch_pdf_tool,  # index=3
            zone_tool,  # index=4
            fallback_tool,  # index=5
        ],
        selector=PydanticSingleSelector.from_defaults(
            verbose=True, prompt_template_str=SELECT_TOOL_PROMPT
        ),
        summarizer=tree_summarizer,  # used when multi-selector is used
        verbose=True,
    )
@Logan M - can you see anything wrong in the setup?
seems fine. Tbh looking at the code though, I'm not sure why predicting an index was chosen, predicting a name should be good enough?

You can do this yourself as well, which maybe is better?

Plain Text
tools_by_name = {tool.metadata.name: tool for tool in tools}

# can pass in a user msg or chat history
resp = llm.chat_with_tools(
  tools, 
  user_msg=f"Given this latest message, pick at least one tool to use: {user_msg}", 
  error_on_no_tool_call=False
)

tool_calls = llm.get_tool_calls_from_response(resp)
for tool_call in tool_calls:
  print(tool_call.tool_name)
  print(tool_call.tool_kwargs)
  tool_result = tools_by_name[tool_call.tool_name](**tool_call.tool_kwargs)
  print(tool_result)
will a PR to do this change in Pydantic Single Selector help?
this is actually what the single selector is doing lol

The above code would let you run the selection yourself, and better handle errors
yeah right, but the Single Selection Class is pretty basic
class SingleSelection(BaseModel):
"""A single selection of a choice."""

index: int
reason: str

Should be changing it to add the name of the tool also, so that it can predict correctly
oh right yea, i did mention that
Add a reply
Sign up and join the conversation on Discord