typically with an assertion error there will be a full traceback that is usually helpful
yea thats what is weird. I think it has to do with the tokenizer_fn in the memory
So there's not an actual traceback to follow?
Are you using the latest version of llama-index? we made some changes recently so that packages are not downloaded at runtime (which breaks aws lambda)
no traceback, i think it has to do with exceeding the context window size, but verifying now..
Try Catch literally only returns AssertionError()
Yea that version should have those changes
I think try catch shallows the traceback though
Unless you specially call this python function thing to print the stack trace
hmm ok i will have to see how to get more info
Well, i tracked it down to the context chat engine, the first prints , the second does not, and for some reason its just returning an AssertionError() , would a try/catch work here or do i need to do that inside of bedrock.py ?
print("LLAMA: All messages " + str(all_messages))
chat_response = self._llm.chat(all_messages)
print("LLAMA: response : " + str(chat_response))Hmm yea probably inside of bedrock.py? There must be something about the API request that's causing issues I feel like
ok let me try that , going to try this in the chat() method
def chat(self, messages: Sequence[ChatMessage], **kwargs: Any) -> ChatResponse:
prompt = self.messages_to_prompt(messages)
completion_response = self.complete(prompt, formatted=True, **kwargs)
try:
return completion_response_to_chat_response(completion_response)
except Exception as e:
print("Error converting completion response to chat response: " + repr(e))
return ChatResponse(response="Error converting completion response to chat response", source_nodes=[])
FWIW i tracked it down in llama_utils.py , this is the assert thats failing , will debug why
if len(messages) > (i + 1):
# if assistant message exists, add to str_message
assistant_message = messages[i + 1]
print("META ASSISTANT MESSAGE: " + str(assistant_message))
assert assistant_message.role == MessageRole.ASSISTANT
print("META ASSISTANT MESSAGE COMPLETE")
While Claude only has this :
# NOTE: make sure the prompt ends with the assistant prefix
if messages[-1].role != MessageRole.ASSISTANT:
messages = [
*list(messages),
ChatMessage(role=MessageRole.ASSISTANT, content=""),
]
str_list = [_message_to_anthropic_prompt(message) for message in messages]
return "".join(str_list)
Those messages have a role of User, not Assistant, which is why the assert is failing
Ah interesting. I think this is just checking that the messages alternate user/assistant?
The good news is you cam override that utils function
Pretty sure it's an input parameter to the LLM
yea just not sure why this one is so much more strict than the others with the assert
the claude one basically just prints whatever is in the messages[] array
And you are right, messages_to_prompt is a param
I think I made that function lol I was copying from llama2's offifical repo
all good wasnt sure if llama was just much more strict or something since the claude one is just a loop and print
