Find answers from the community

Updated 2 months ago

Llama2

I'm using llamaindex to answer questions (with lama2 70b) with data from a confluence. I first tested pdf exports and saw "okay" results. trying to improve this, I switched to html pages, using UnstructuredReader as parser/extracter.
however now my responses are really bad, questions that where easy at first are now not answered with garbage or "I dont know"
how could I fix this?

5 comments

LLogan M

Have you tried inspecting the prompts being sent to the LLM? And debugging using those and the LLM directly?

You can see LLM inputs/outputs using this
https://gpt-index.readthedocs.io/en/stable/end_to_end_tutorials/one_click_observability.html#simple-llm-inputs-outputs

LLogan M

Maybe you arent formatting queries with the proper [INST] tokens for llama2

CCHY4E

I will try that, but what do you mean with [INST] tokens :ablobnervous2:

LLogan M

Llama2 has pretty specific formatting requirements 😅

How are you using llama2? With Huggingface? Replicate?

CCHY4E

using Ollama and llamaindex connects to it

Add a reply