Find answers from the community

Updated last year

I have built an RAG chatbot using "meta-

At a glance
I have built an RAG chatbot using "meta-llama/Llama-2-7b-chat-hf" model to let model act as a document question answering chatbot and below are the model params

llm = HuggingFaceLLM(
model_name="meta-llama/Llama-2-7b-chat-hf",
tokenizer_name="meta-llama/Llama-2-7b-chat-hf",
query_wrapper_prompt=PromptTemplate("<|system|>\n</s>\n<|user|>\n{query_str}</s>\n<|assistant|>\n"),
context_window=3900,
max_new_tokens=256,
model_kwargs={"quantization_config": quantization_config},
# tokenizer_kwargs={},
generate_kwargs={"temperature": 0.3, "top_k": 50, "top_p": 0.95},
messages_to_prompt=messages_to_prompt,
device_map="auto",

)

I have used VectorStoreIndex to index my documents. When I query my model it is able to give answers related to the document accurately( Which is a good thing), but it is also answering questions that are out of my documents, I want to limit it to answer questions only related to my docs, How can I do that?
T
C
8 comments
Have you included that instruction in the prompt? But in general with the smaller and less capable models they will still be prone to hallucinating in that situation
Oh so stop that hallucination, should we include like “please answer to questions only that are related to the documents” in the prompt?
Yeah I included in the instruction prompt to strictly stick to the source documents, it got better than before but still it is answering some things out of document, should I give a more enhanced instruction prompt??
Yeah including it in the prompt is good but with that model I'm not sure you can get it 100% perfect
I’m very much new to llms, prompt engineering and llama-index can you guide me any open source model that would satisfy my requirement?
I mean you can try working on the prompts
But here's the leaderboard for the open source ones: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
Add a reply
Sign up and join the conversation on Discord