Find answers from the community

Updated 3 months ago

Hi, with ContextChatEngine is there any

Hi, with ContextChatEngine is there any reliable way to get answers only from the context? Pretty often, I face the situation when it can't find the relevant data in the context and takes it from its generic knowledge what is usually pretty wrong. Thanks.
Plain Text
retriever = index.as_retriever(verbose=True, chat_mode="context", similarity_top_k=similarity_top_k)
# CustomContext inherits the ContextChatEngine, and just has a small changes for the best results.
custom_chat_engine = CustomContext.from_defaults(
                                    retriever=retriever,
                                    memory=chatmemory, 
                                    context_template=generate_context_template(),
                                    system_prompt=prepared_system_prompt,
                                    node_postprocessors=[CustomPostprocessor(
                                            context_limit, query_text + prepared_system_prompt, project.db_name, None)])
response = custom_chat_engine.chat(query_text, chat_history=chat_history)
a
S
2 comments
can you share how did you create a custom context engine?
It's just rewrites one method:
Plain Text
DEFAULT_CONTEXT_TEMPLATE = (
    "Context information is below."
    "\n--------------------\n"
    "{context_str}"
    "\n--------------------\n"
)
class CustomContext(ContextChatEngine):
    def _get_prefix_messages_with_context(self, context_str: str) -> List[ChatMessage]:
        """Get the prefix messages with context."""
        # ensure we grab the user-configured system prompt
        system_prompt = ""
        prefix_messages = self._prefix_messages
        if (
            len(self._prefix_messages) != 0
            and self._prefix_messages[0].role == MessageRole.SYSTEM
        ):
            system_prompt = str(self._prefix_messages[0].content)
            prefix_messages = self._prefix_messages[1:]

        context_str_w_sys_prompt = system_prompt.strip() + context_str # Opporsite order <------- the only difference
        return [
            ChatMessage(content=context_str_w_sys_prompt, role=MessageRole.SYSTEM),
            *prefix_messages,
        ]
Add a reply
Sign up and join the conversation on Discord