I am using ```chat engine index as chat

I am using

Plain Text

chat_engine = index.as_chat_engine(
                    chat_mode="context",
                    system_prompt="You are a chatbot with access to extra context information about dogs. Only answer questions if you can find the answer in the context, otherwise, kindly inform the user that you cannot find the answer.",
                    chat_history=custom_chat_history
    )

All is working well and the LLM only answers questions about "dogs". My issue is that when I introduce the recent chat history, if someone asks a question about "cats", the LLM properly handles the quesiton the first try but any follow-up question including cats will be answered. Any recommendations for handling this situation?

11 comments

eelmegatan26

@Logan M I could use another bail out on this one 😉 My guess is that context mode is counting the chat history as part of the context?

LLogan M

Yea, the chat history is part of the context. But isn't that the main use case for using a chat engine?

eelmegatan26

I want the Chat to have context of what we're discussing so if there is a follow up question e.g. Where do THEY live?, Who is SHE? etc. The chat can answer with that knowledge.

eelmegatan26

My problem is that once the topic is introduced into the chat history the system prompt to not answer questions about X gets ignored.

LLogan M

hmmm, I see

LLogan M

Maybe the system prompt just needs more verbose instructions? 😅

eelmegatan26

I'm trying I promise! 😉

eelmegatan26

Under the hood is there a way I can reference "context" vs. "chat history" that the context mode can interpret?

LLogan M

hmm.. I think chat history could be referenced as previous messages ? And context is already directly called context under the hood 👀