Find answers from the community

Updated 2 months ago

When streaming a chat with a react agent

When streaming a chat with a react agent i get some strange results:
Plain Text
def get_react_agent(
    vector_store: PineconeVectorStore,
    history: List[ChatMessage],
    user_id: str,
    model: ChatLLM,
) -> ReActAgent:
    query_engine = get_query_engine(
        user_id=user_id, vector_store=vector_store, model=model
    )
    query_engine_tool = QueryEngineTool.from_defaults(query_engine=query_engine)

    return ReActAgent.from_tools(
        tools=[query_engine_tool],
        llm=model.value,
        chat_history=history,
    )
           
agent = get_react_agent(
        vector_store=get_pinecone_vector_store(),
        history=history,
        user_id=user_id,
        model=model,
    )
# Valid types based on looking at source code
response = agent.stream_chat(message)  # pyright: ignore

... generator websocket push code ...

print(response.response)
#<function_calls>
#<invoke>
#<tool_name>query_engine_tool</tool_name>
#<parameters>
 # "input": "hello"
#</parameters>
#</invoke>
#</function_calls>

#Thought: I need to use the query_engine_tool to help me understand and respond to the user's input.

#Action: query_engine_tool

#Action Input: {"input": "hello"}
L
m
15 comments
lol seems like the LLM tried to write some xml?
Definitely a symptom of the LLM being used
yeah I guess the quality of anthropic has fallen greatly
anthropic really does like XML from my experience
We almost need an XML mode for these kinds of structured outputs
So much work though lol
Yeah that's weird, gonna try and chat with claude without ReAct and see if it's better
Not as easy as it seems to get multimodel chat working across everything
multimodal support is pretty jank at the moment -- hopefully that improves
yeah im gonna have to get gemeni in soon here
@Logan M I am seeing under the hugging face class:
Plain Text
    def stream_chat(
        self, messages: Sequence[ChatMessage], **kwargs: Any
    ) -> ChatResponseGen:
        raise NotImplementedError

I kind of want to implement this, any good examples of where to look for best practices inside the codebase
πŸ‘ ill see what i can do, thanks!
Add a reply
Sign up and join the conversation on Discord