Find answers from the community

Updated 2 months ago

Llama2

I'm using llamaindex to answer questions (with lama2 70b) with data from a confluence. I first tested pdf exports and saw "okay" results. trying to improve this, I switched to html pages, using UnstructuredReader as parser/extracter.
however now my responses are really bad, questions that where easy at first are now not answered with garbage or "I dont know"
how could I fix this?
L
C
5 comments
Have you tried inspecting the prompts being sent to the LLM? And debugging using those and the LLM directly?

You can see LLM inputs/outputs using this
https://gpt-index.readthedocs.io/en/stable/end_to_end_tutorials/one_click_observability.html#simple-llm-inputs-outputs
Maybe you arent formatting queries with the proper [INST] tokens for llama2
I will try that, but what do you mean with [INST] tokens :ablobnervous2:
Llama2 has pretty specific formatting requirements πŸ˜…

How are you using llama2? With Huggingface? Replicate?
using Ollama and llamaindex connects to it
Add a reply
Sign up and join the conversation on Discord