Find answers from the community

Updated last year

Return

@Logan M So the OpenAI Agent routes a query to a query engine tool via function calling, but once it gets the output, it also does some response synthesizing of its own. Is there anyway to just do no response synthesizing step and just spit out the output of the tool? I understand something like RouterQueryEngine can help here, but I suppose the caveat is it doesn't use OpenAI function calling directly I suppose like an agent? Also, it normally doesn't have memory like an agent. What are your suggestions? Trying to cut down on latency here as much as I can, with a system of agent -> sub agents -> tools.

I tried avoiding the agent -> sub agents using an ObjectIndex for tools too, but even that still has a response step after getting output from a tool, that seems to take a good amount of time.
L
V
10 comments
I think we've been meaning to add a "return direct" option to tools, but it hasn't happened yet πŸ˜…
Is there an open PR/issue for this?
If not, where can I start? Would love to contribute here πŸ™‚
Hmm not that I know of πŸ‘€ there might be a github issue somewhere, definitely not an open PR I've seen yet

I think a PR would involve two parts

  1. Adding return_direct as an attribute to the base tool class (or it could be in the tool metadata, unsure πŸ€” )
https://github.com/run-llama/llama_index/blob/80484a4669cbd2f09a206b82d594add8f967695d/llama_index/tools/types.py#L67

  1. Updating the openai and react agents to respect that new attribute in the agent loops
Somewhere around here (but also the async version too)
https://github.com/run-llama/llama_index/blob/80484a4669cbd2f09a206b82d594add8f967695d/llama_index/agent/openai_agent.py#L293

And here-ish?
https://github.com/run-llama/llama_index/blob/80484a4669cbd2f09a206b82d594add8f967695d/llama_index/agent/react/base.py#L241
Tbh I've been meaning to add this. Just on holidays still and away from my laptop lol
Thank you so much for this!
Will get started working on this and raise a PR
Enjoy your holidays, Logan!
And thank you for being here to respond during your break! :)
Also @Logan M What happens in the case of multiple tools being accessed?
Add a reply
Sign up and join the conversation on Discord