Find answers from the community

Updated 2 months ago

Hey guys I am encountering a weird issue

Hey guys, I am encountering a weird issue. I am using the context chat engine and for most of my messages I get great replies, but for some (and pretty specific one, like "how to hold an order") I get the following error:


Exception in thread Thread-18 (process_event):
Traceback (most recent call last):
File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py”, line 1038, in _bootstrap_inner
self.run()
File “/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py”, line 975, in run
self._target(*self._args, **self._kwargs)
File “/Users/tigran/Desktop/Project llama/chat.py”, line 417, in process_event
response = chat_engine.chat(text)
^^^^^^^^^^^^^^^^^^^^^^
File “/Users/tigran/Desktop/Project llama/venv/lib/python3.11/site-packages/llama_index/chat_engine/context.py”, line 125, in chat
prefix_messages = self._get_prefix_messages_with_context(context_str_template)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/Users/tigran/Desktop/Project llama/venv/lib/python3.11/site-packages/llama_index/chat_engine/context.py”, line 114, in _get_prefix_messages_with_context
context_str = context_str_template.format(system_prompt=system_prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: ‘“content”’
W
T
L
9 comments
That's strange! What is the version that you are trying with?
Latest one from Github
I could fix it minutes ago with this inside _get_prefix_messages_with_context:

context_str = context_str_template.replace("{system_prompt}", system_prompt)
return [ChatMessage(content=context_str, role="system")] + prefix_messages
but I'm not a professional engineer tbh, so you might have better solutions, I used nice prompt engineering with ChatGPT to address the issue.
It should be like this only.
context_str = context_str_template.format(system_prompt=system_prompt)

If it is working in some responses and for some cases it is throwing error. I believe there could be some missing part in the library.


Can you share your code. I can try to check
Yea would be nice if this was repoducabile locally 🤔 Seems very weird
So, I am working on a Slack app.

This is my chat.py:

def initialize_chat_engine(user_id):
yerevan_project_index, project_name = user_engines.get(user_id, (None, None))
if yerevan_project_index is None:
raise ValueError("Yerevan account's settings have not been configured for this user.")

## CONTEXT MODE
memory = ChatMemoryBuffer.from_defaults(token_limit=2000)
chat_engine = yerevan_project_index.as_chat_engine(
chat_mode="context",
memory=memory,
service_context = service_context,
refine_template=DEFAULT_REFINE_PROMPT,
system_prompt=(f'Your name is Yerevan, who assisting only with questions related to {project_name}')
response_synthesizer=response_synthesizer,
similarity_top_k=5
)
return chat_engine
This is my slack application's snippet were messages are handled:

def process_event(data):
logging.debug("Processing event...")
text = data['event'].get('text')
channel_id = data['event'].get('channel')
user_id = data['event'].get('user')

if not text or not channel_id or not user_id:
return


user_workspace = get_workspace_name(user_id)
user_name = get_user_name(user_id)

with user_data_lock:
if user_id not in user_data:
message_text = "Please connect to your Yerevan account."
try:
client.chat_postMessage(channel=channel_id, text=message_text)
except SlackApiError as e:
logging.error(f"Slack API error occurred: {e}")
return
else:
cipher_suite = Fernet(user_data[user_id]['key'].encode('utf-8'))

try:
chat_engine = initialize_chat_engine(user_id)
except Exception as e:
logging.error(f"Error occurred while initializing chat engine: {e}")
return

response = chat_engine.chat(text)
logging.debug(f"Response: {response}")

if isinstance(response, AgentChatResponse):
message_text = response.response
else:
message_text = "Unexpected response type."


record_interaction(user_workspace, user_name, text, message_text)

try:
client.chat_postMessage(channel=channel_id, text=message_text)
except SlackApiError as e:
logging.error(f"Slack API error occurred: {e}")
ChatGPT's response that worked for me:

Thank you for providing the output. It seems that the context_str_template looks correct with the appropriate placeholders {context_str} and {{system_prompt}}. The error is likely happening because of how the format method is used in the _get_prefix_messages_with_context function.

In Python, when you have double curly braces {{}} in a format string, it's used to escape the curly braces and treat them as literal characters in the formatted string. So, {{system_prompt}} becomes {system_prompt} in the formatted string. When you later try to format the context_str_template again, Python is looking for a key named system_prompt in the template string. This seems to be causing the issue.

You can modify the code to handle this by using the formatted string once, and replace the placeholder with the actual system prompt. Here's the modified _get_prefix_messages_with_context method:

Plain Text
def _get_prefix_messages_with_context(
    self, context_str_template: str
) -> List[ChatMessage]:
    """Get the prefix messages with context"""

    # ensure we grab the user-configured system prompt
    system_prompt = ""
    prefix_messages = self._prefix_messages
    if (
        len(self._prefix_messages) != 0
        and self._prefix_messages[0].role == "system"
    ):
        system_prompt = str(self._prefix_messages[0].content)
        prefix_messages = self._prefix_messages[1:]

    # Replace the placeholder with the actual system prompt
    context_str = context_str_template.replace("{system_prompt}", system_prompt)
    return [ChatMessage(content=context_str, role="system")] + prefix_messages


Try making this change and test the code again to see if the error persists.
Add a reply
Sign up and join the conversation on Discord