Agent

BBP

I've followed the chatbot tutorial, and now I'm changing it to use my own txt documents. It's having a very hard time distinguishing whether to use a tool or not--I believe this happens because some of the information in my txt document is too generic and isn't esoteric enough, i.e., the LLM is confusing my documents with information in its own database

Is there a way to force the agent_chain to use my tools on every run?

23 comments

LLogan M

The age old question

Not an option to force at the moment. But whether or not it uses a tool is based almost entirely on the tool description.

BBP

to be clear:

Plain Text

    tool_config = IndexToolConfig(
        query_engine=query_engine, 
        name=f"Vector Index {y}",
        description=f"useful for when you want to answer queries about the {y} quiz",
        tool_kwargs={"return_direct": True}
    )

this line?

Plain Text

        description=f"useful for when you want to answer queries about the {y} quiz",

BBP

If so, is a shorter or longer description better?

BBP

Bless ❤️

LLogan M

Yea that's the line.

Maybe try to make it a bit more descriptive if you can? I know it's hard to do if you are creating these automatically lol

"Useful for when you need information about the {y} quiz, which includes topics like blah blah blah"

LLogan M

That's maybe a bad example, but you get what I mean I think

BBP

That makes sense! Thank you ser! ❤️

BBP

hey @Logan M

I have 3 simple txt documents, the 1st contains the Canada Quiz questions and answers and the next 2 are actually filled out quizes with answers from students.

I'm trying to get the agent_chain to actually look at the rubric, and then calculate how many correct answers/points each student got.

I'll prompt the agent_chain like so:

Plain Text

agent_chain.run(input="""                                        
Tell me about the Canada Quiz of 2023 administered by Botany Downs Secondary College. 
""")

agent_chain.run(input="""                                        

Based on your understanding of the question and answer rubric in the Canada Quiz of 2023 administered 
by Botany Downs Secondary College, 
How many answers did Bob Heart get correct in the Canada Quiz of 2023?""")

BBP

But based on the answers, it looks to me like the agent_chain is flushing its memory after every run

BBP

Answers here:

Plain Text

> Entering new  chain...

Thought: Do I need to use a tool? No
AI: The Canada Quiz of 2023 administered by Botany Downs Secondary College is an annual quiz competition that tests students' knowledge of Canadian history, geography, and culture. It consists of multiple-choice questions and is open to all students in grades 9-12. The quiz is divided into two parts: the first part is a general knowledge quiz, while the second part is a more in-depth exploration of Canadian history, geography, and culture. The quiz is designed to challenge students and help them develop a deeper understanding of Canada.

> Finished chain.


> Entering new  chain...

Thought: Do I need to use a tool? Yes
Action: Graph Index
Action Input: Bob Heart> Current query: Bob Heart
> New query: What is the significance of Bob Heart in the context of the Canada Quiz number 0?
> Current query: Bob Heart
> New query: What is the significance of Bob Heart in the context of the Canada Quiz number 0?
> Current query: Bob Heart
> New query: Who is the main character of the Canada Quiz number 1?
> Current query: Bob Heart
> New query: Who is the main character of the Canada Quiz number 1?
> Current query: Bob Heart
> New query: What is the significance of Bob Heart in the Canada Quiz number 2?
> Current query: Bob Heart
> New query: What is the significance of Bob Heart in the Canada Quiz number 2?

Observation: Based on the context information provided, Bob Heart is a student who participated in the Canada Quiz administered to students of Botany Downs Secondary College in the year 2023.

BBP

is there an agent_chain that can persist its memory, or a way to make it persist its memory for a set number of .run() calls?

BBP

I fed these documents into chatGPT3.5 and 4.0 and it worked great so I know it's possible 😄 kekeke

BBP

Totally understand this is a technical challenge and any help at all is much appreciated ❤️ 🙏 👑

LLogan M

Not a langchain expert, but any reason to not use an actual agent instead of an agent chain? Then you can add a memory module to the agent

BBP

The reason is: I legit don't know the difference between an agent v. an agent chain 😂

BBP

Sorry about that fam, this is all new to me

BBP

I'll look into using an agent--do you have any doc recommendations? 🙏

LLogan M

No worries! It's pretty confusing haha

For langchain agents, you can start around here

https://docs.langchain.com/docs/use-cases/personal-assistants

BBP

aeee thank you thank you ser

BBP

that link takes me to Use Cases/Personal Assistants but when I click on 'Python Guide' here:

Plain Text

!INFO
Python Guide

JS Guide

BBP

I just get a 'page not found'-- is this the same for you? Is there actually a full on python guide somewhere?

LLogan M

Oh that's annoying. This link should leD to some similar info

https://python.langchain.com/docs/modules/agents/

BBP

thank you ser! ❤️

Add a reply

Find answers from the community

Agent