Find answers from the community

Updated 3 months ago

With chat agents as the llm e

With chat agents as the llm (e.gChatOpenAI ), if using refine method , would it keep an ongoing chat history on node 1 - N? I have seen turbo 3.5 do pretty bad as it refines each node but I had a creeping suspicion the chat history was just adding accumulating too much context as the refine process went on. At least for some of the azure models I am limited to using the ChatOpenAI model . And response mode was set to “refine” fyi
L
1 comment
the chat history doesn't accumulate with refine

It just shows the LLM the existing answer, the user query, and the new context, and asks the LLM to either repeat or update the existing answer

In most chat engines, the query engine is disconnected from the actual chat history
Add a reply
Sign up and join the conversation on Discord