Find answers from the community

Updated 3 months ago

Using OpenInferenceCallbackHandler with condense_plus_context chat engine

Has anyone ever used OpenInferenceCallbackHandler with condense_plus_context chat engine? In the log, the response text is the output of my DEFAULT_CONDENSE_PROMPT. However, in the chat menu, I see to the response to the system prompt.
W
a
5 comments
You mean you see soemthing different in the log and something else in the response object?
Yes, as far as I understand, for condense plus context. First uses history and query to extend the prompt with llm call. This is when the OpenInferenceCallbackHandler updates the log.

However, then it uses retrieved nodes and system prompts to generate responses in chat. The output of chat_engine is the response. However the log is condense prompt output
Yes as it logs the condense question part only
But I thought OpenInference callback handler does not retrieve query info from the logs. What I referred to as logs was the query data.
Add a reply
Sign up and join the conversation on Discord