Find answers from the community

Updated 2 months ago

my prompt template just ends in `return

my prompt template just ends in return json. I use the {context_str} and {query_str} correctly I think, they are being included in the prompt output that goes over the wire.

15 comments

nnbulkz

I'm looking at anthropic_utils.py and see the prompt pieces, is there an easy way to override this?

nnbulkz

a simple way to override ASSISTANT_PREFIX would be valuable

nnbulkz

https://docs.anthropic.com/claude/docs/prefill-claudes-response

nnbulkz

looks like the answer is messages_to_prompt but I'm looking for the right syntax here, keep getting an error saying instance of MessagesToPromptType

nnbulkz

(obscure pydantic error here)

nnbulkz

here's my code,

nnbulkz

Plain Text

..
from llama_index.llms import ChatMessage
..

llm = Bedrock(model="anthropic.claude-instant-v1",
temperature=0.001,
profile_name=PROFILE_NAME,
max_tokens=max_output_tokens,
context_size=max_input_tokens,
messages_to_prompt=[ChatMessage(role="assistant", content=' {"answers":')]
)

nnbulkz

@Logan M hate to ping you, but any idea?

nnbulkz

llama-index 0.9.44

LLogan M

messages_to_prompt should be a callable

LLogan M

Plain Text

def messages_to_prompt(messages):
  return "\n".join([str(x) for x in messages])

LLogan M

something like that

nnbulkz

ah ok thanks

nnbulkz

now I am getting prompt must start with "Human:" turn after an optional system prompt trying to override the messages_to_prompt

LLogan M

nice!

Add a reply