In the huggingface LLM, just set the query wrapper prompt. It will look something like this
BOS, EOS = "<s>", "</s>"
B_INST, E_INST = "[INST]", "[/INST]"
B_SYS, E_SYS = "<<SYS>>\n", "\n<</SYS>>\n\n"
query_wrapper_prompt=(
f"{BOS}{B_INST} {B_SYS}{system_prompt_str.strip()}{E_SYS}"
f"{completion.strip()} {E_INST}"
)
If you aren't using a system prompt, then it would look like this
query_wrapper_prompt=(
f"{BOS}{B_INST} "
f"{completion.strip()} {E_INST}"
)
Tbh we should probably align the interface though