A community member is experiencing issues with hallucination when using the 4bit llama2 70b model. They are seeking advice on how to prompt better or finetune the model. Other community members suggest using the INST and EOS/BOS tokens when prompting, as the format for llama2-chat is quite strict. They provide a sample format and mention that there are utility functions available in the llama_index library that may be helpful for implementing a custom LLM class.
