Quality

The agent has to write its own query, and sometimes it may not even use the tool

Maybe that's why?

not exactly, I have the tool and it actually detects it and uses it however the answer that I receive its much worse in quality

I dont know if you want me to show code or if you could have a quick call just to show you, I can even pay you for your time and help since I dont really know how to solve this issue

Sorry, just boarding a flight right now LOL

I'm not really sure tbh. There isn't really a different between the agent tool and querying directly, at least code wise.

Not sure what's up otherwise... if you are setting any query kwargs, maybe they aren't making it to the agent index query properly? 👀

haha it happens

well I dont think its a problem with the index

this is just simple example of how I load the index for the query

first ss is for the normal query

the other ones are building the tools and having the indexes for the agent

and in the action it selects the actual tool

but then answer quality drops a lot

In the tools, you aren't setting the top k or response mode in the query calls (you used these settings in the very first screenshot). Maybe adding that will help the response quality?

I will add it but I dont think it will help, because it kinda does access the right document

but then gives a very generic answer instead of an specific one

but its worth a shot

Especially the top k, I think it will help quite a bit 👀

ok, will try and keep you updated for this, btw have a great flight!

Couldnt really get a better answer, but I was wondering if It Is posible to sabe only the action to pass It into the query directly, (so using the agent only to choose the right tool)

Hmmm I'm not sure on that one. I'd have to read up on the langchain docs

or is there any other way to select a tool but not modify original query?

In the tool description, you could try and give some instruction to not modify the user message if that tool is chosen. I've had some success with instructions like this in tool descriptions

but for example, I tried this "give me a description of tana"
and I put in the tool description "DO NOT MODIFY USER MESSAGE AFTER SELECTION THIS TOOL"

then when I run the agent i get this

Entering new AgentExecutor chain...
Thought: I need to find information about "tana" in the available tools.
Action: Quip Documents
Action Input: {"search_query": "tana"}

dont really understand how to modify this or do it so it wont modify user input

Lol yea so the LLM just decided to ignore your description I guess 😅

Maybe the name of the tool is causing some bias in how it searches (normally, a search query should be short for traditional search engines, which is what it seems to be doing)

Could try changing the name of the tool to "user Q&A" or something maybe?

Langchain is so finicky with this stuff, since a lot of the decision making is up to the LLM

yeah but the name of the tool wouldnt it only be to identify where the information is stored? and well, it almost always finds the correct tool to use

Right, but I think the name of the tool has some influence on how it behaves

The LLM just reads all the tool names and descriptions, and has to write which tool to use and the input.

So both the names and description have influence... very annoying haha

yeah its very annoying and cant really make it work without it changed it haha

but I guess thats only on the llm then?

if there another way to do some rerouting for tools without agents