LlamaIndex
Log in

Find answers from the community

Updated last year

Prompting

wwoojim
·
Yeah from experience using 4bit llama2 70b it's hallucinating pretty badly. Gotta figure out how to prompt better or to finetune it
L
w
7 comments
LLogan M
Are you using the INST and EOS/BOS tokens when prompting?
LLogan M
The format for llama2-chat is pretty strict I've found
wwoojim
Did you mean this?
wwoojim
<s>[INST] <<SYS>>
{{ system_prompt }}
<</SYS>>

{{ user_message }} [/INST]
wwoojim
I was planning to look into incorporating this into llama index calls to a llama 2 api using this format
wwoojim
Not sure if llama index already has this supported format in the code
LLogan M
We have some utils functions that help with this. If you are implementing a custom llm class, then they will be useful

https://github.com/jerryjliu/llama_index/blob/main/llama_index/llms/llama_utils.py
Add a reply
Sign up and join the conversation on Discord