Find answers from the community

Updated 2 months ago

Agent

Hi there! I've been using the ContextRetrieverOpenAIAgent.from_tools_and_retriever context agent. I noticed that it's not managing the number of tokens effectively, and as a result, it often exceeds the limit. Shouldn't this context agent inherently control the number of tokens?
L
C
17 comments
Is it running out of tokens because of the chat history? Or something else?
FYI better managed chat history is coming soon πŸ™
Yeah, it's because of the chat history
In the meantime, you can manually manage the chat history by accessing the list in agent._chat_history. It's just a list of ChatMessage objects right now
Got it! I'll implement that.
I have another question. I've pre-added an initial message to the chat_history property of from_tools_and_retriever which includes a question for my chat. However, when the user responds to this question with just "yes", it calls the function with an empty input, thereby causing an error:
Plain Text
Calling function: my-index with args: {
  "input": ""
}

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/llama_index/embeddings/openai.py", line 106, in get_embedding
    return openai.Embedding.create(input=[text], model=engine, **kwargs)["data"][0][
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openai/api_resources/embedding.py", line 33, in create
    response = super().create(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./context-pinecone.py", line 165, in <module>
    response = context_agent.chat(text_input, chat_history=messages)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/llama_index/agent/context_retriever_agent.py", line 165, in chat
    return super().chat(formatted_message, chat_history=chat_history)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
could you help me with that?
hmmm.. do you have some code that I could run to reproduce this?
sure, I just created this example:
When you send 'Yes', it sometimes calls the function with an empty string, and at other times it calls it with 'Yes'. Therefore, it is intermittent.
Attachment
Screenshot_2023-07-11_at_12.48.18_PM.png
Attachment
Screenshot_2023-07-11_at_1.17.11_PM.png
I think it might help to modify your tool descriptions slightly. It's not clear from the description that any input is needed

Try this

Plain Text
query_engine_tools = [
    QueryEngineTool(
        query_engine=engine_home,
        metadata=ToolMetadata(
            name="llamaindex-home",
            description=f"Useful for asking questions about LLamaIndex",
        ),
    ),
    QueryEngineTool(
        query_engine=engine_openai_agent,
        metadata=ToolMetadata(
            name="llmaindex-openai_agent",
            description=f"Useful for asking questions about Context-Augmented OpenAI Agent",
        ),
    )
]


I'm assuming you are planning to add more information to the indexes eventually? If not, you might be better off using a function tool that just returns the string info, since it's pretty short
Thanks! That solves the problem
Add a reply
Sign up and join the conversation on Discord