LlamaIndex
Log in

Find answers from the community

Updated 8 months ago

llama_index/llama-index-integrations/llm...

xxrt
·
Hello, I have a question for Bedrock llms usage

I notice in new 0.10.26 release that for Anthropic adding HUMAN/ASSISTANT prefix is added from library

HUMAN_PREFIX = "\n\nHuman:"
ASSISTANT_PREFIX = "\n\nAssistant:"

https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/llms/llama-index-llms-bedrock/llama_index/llms/bedrock/utils.py#L24

So I supose that I no longer need to add anything from my side ?


And second question, for Mistral do I still need to add prefix "<s> [INST]" <prompt "[/INST]" or is no longer required ?

Thank you
W
1 comment
WWhiteFang_Jr
  1. yes if it is added, then yes no need to add from your side.
  1. For Open-source llms, the prefix are always diff and it is hard to keep all of it. You will have to add it by yourself if it is not already present.
Add a reply
Sign up and join the conversation on Discord