I am building a multi tool AI agent with 3 different tools, including QueryEngine Tools and a Function Tool, but my agent answers queries but doesn't return any answer, instead diverging from the original query.

The community member is building a multi-tool AI agent with 3 different tools: 2 QueryEngine Tools and one Function Tool. They used llama_index evaluation metrics to measure their model's performance, which was good. However, the issue they are facing is that their agent answers the given query but does not return any answer, instead it diverges from the original query, either hitting the maximum iteration or providing no response.

In the comments, another community member suggests that the issue may be due to the way the output parser checks for "Action:" before "Answer:" in the output, which may not detect the end of the react loop. They recommend trying a different LLM, as open-source LLMs may not work well for agents. Alternatively, they suggest writing a custom output parser and passing it in.

The original community member responds that they have limited options as they are using a Microsoft LLM capped with a Virtual Language Model (vllm) for better performance, and they are working on a Colab free tier GPU. They mention trying to make a custom output parser and ask for other suggestions, such as decreasing the max_new_tokens or fine-tuning the LLM on the data.