It calls a query engine, and then the LLM interprets the response from the query engine in context of the chat history
hey, just wondering, what are you using to create this visualization @ChicoButico ?
@theoxd Phoenix... There are a couple of jup notebooks in the llamaindex project that show how to use it.
I was getting crazy with debug level logs and then I found it. it makes all the difference
Hi, I am also using OpenAIAgent.from_tool to build chatbot. Do you happen to know how to customize the agent default prompt?
you can pass a string system_prompt argument to from_tools
@Logan M is there a way of instructing the agent to not make this last LLM call? The tool is usually returning a good response but then this last LLM call happens to be hallucinating quite frequently.
My assistant is in portuguese, but I think you might see what's happening in the image without having to translate it
The tool responds the steps I need and then the last LLM call returns the output like "if you need anything more from me, let me know", which end up to be the very last response without my user getting the steps he asked about...
There isn't yet. Been meaning to add some option to "return direct" from tools. But this wouldn't be dynamic, if you set it, it would always return directly
I haven't found a way of configuring this last LLM call also, like passing a system prompt instructing the to use the exact same tool message.
I had to go back to chatmode condense_plus_context while I don't figure out a way of making this customizations to the agent.
thanks for you reply and at some point when things get a bit calmer at work, I plan to start contributing to llama_index. This project is awesome.
Yea the last LLM call should technically already be using the system prompt, if you passed it in 🤔 OpenAIAgent.from_tools(..., system_prompt="...")
Or, at least I thought it did haha
You're definetely right. Oh gosh, I'm so noob. Thanks!
Hi, did you happen to get any luck to control the last LLM call ? I tried to tweak system_prompt, but it still sometimes cannot return the 'correct' responses while query tool does
@rachel_001. I wasn't... I ended up stopping using agents for now and rolledback to use a query pipeline. I didn't study how to implement custom agents yet and that would probably solve the issues I had with it, but I'm on a schedule to deliver something and the query pipeline was just serving me well enough.
My query pipeline goes like:
I first do a manual call to the LLM using the default prompt of CondenseQuestionChatEngine (I don't use the engine directly because I don't want to query the index with it) and then after that I call my tool, which will have that condensed customer query as an input and will route to whatever query engine fits best.
By doing this I'm getting only the tool result and am also saving that last LLM call, that was the main cause of hallucination in my agent.
