Agent

CCesar

Hi there! I've been using the ContextRetrieverOpenAIAgent.from_tools_and_retriever context agent. I noticed that it's not managing the number of tokens effectively, and as a result, it often exceeds the limit. Shouldn't this context agent inherently control the number of tokens?

17 comments

LLogan M

Is it running out of tokens because of the chat history? Or something else?

LLogan M

FYI better managed chat history is coming soon 🙏

CCesar

Yeah, it's because of the chat history

LLogan M

In the meantime, you can manually manage the chat history by accessing the list in agent._chat_history. It's just a list of ChatMessage objects right now

CCesar

Got it! I'll implement that.

CCesar

I have another question. I've pre-added an initial message to the chat_history property of from_tools_and_retriever which includes a question for my chat. However, when the user responds to this question with just "yes", it calls the function with an empty input, thereby causing an error:

CCesar

Plain Text

Calling function: my-index with args: {
  "input": ""
}

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/llama_index/embeddings/openai.py", line 106, in get_embedding
    return openai.Embedding.create(input=[text], model=engine, **kwargs)["data"][0][
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/api_resources/embedding.py", line 33, in create
    response = super().create(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./context-pinecone.py", line 165, in <module>
    response = context_agent.chat(text_input, chat_history=messages)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/llama_index/agent/context_retriever_agent.py", line 165, in chat
    return super().chat(formatted_message, chat_history=chat_history)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

CCesar

could you help me with that?

LLogan M

hmmm.. do you have some code that I could run to reproduce this?

CCesar

sure, I just created this example:

CCesar

When you send 'Yes', it sometimes calls the function with an empty string, and at other times it calls it with 'Yes'. Therefore, it is intermittent.

CCesar

Attachment

Screenshot_2023-07-11_at_12.48.18_PM.png

CCesar

Attachment

LLogan M

ohhh I see

LLogan M

I think it might help to modify your tool descriptions slightly. It's not clear from the description that any input is needed

Try this

Plain Text

query_engine_tools = [
    QueryEngineTool(
        query_engine=engine_home,
        metadata=ToolMetadata(
            name="llamaindex-home",
            description=f"Useful for asking questions about LLamaIndex",
        ),
    ),
    QueryEngineTool(
        query_engine=engine_openai_agent,
        metadata=ToolMetadata(
            name="llmaindex-openai_agent",
            description=f"Useful for asking questions about Context-Augmented OpenAI Agent",
        ),
    )
]

I'm assuming you are planning to add more information to the indexes eventually? If not, you might be better off using a function tool that just returns the string info, since it's pretty short

CCesar

Thanks! That solves the problem

LLogan M

Perfect!

Add a reply

Find answers from the community

Agent