Responw

jjma7889

I am using retriever with top k = 10 and a generic responseSynthesizer, i can see the result is in node 5, however no matter what mode I use (I tried all of them), the result is always wrong (n/a mostly, and a couple of wrong answers). But, if I use a keyword post processor to narrow the results to 1 node, then the same responseSynthesizer returns the correct answer. I suspect if I ask LLM with each node, it will have one correct answer. Not sure why llama index synthesizer mostly gives N/A.
What should I do to improve the responseSynthesizer quality?

7 comments

LLogan M

You can try a different response mode ("compact" is default, there's also "refine" and "tree_summarize")

You could also use a reranking step to narrow down the top k

jjma7889

i tried tree_summarize, i tried all of them listed

LLogan M

Probably then I would use a reranker. 10 nodes is a lot of text to get lost in for most LLMs

jjma7889

Plain Text

retriever = VectorIndexRetriever(
    index=index, similarity_top_k=10, response_mode="tree_summarize"
)

retriever_results = retriever.retrieve(
    CRITERION_PERIOD_OF_PERFORMANCE_LLM_RETRIEVE_PROMPT
)
response_synthesizer = get_response_synthesizer(
    structured_answer_filtering=True
)

response_synthesizer_results = response_synthesizer.synthesize(
    CRITERION_PERIOD_OF_PERFORMANCE_LLM_QUESTION, retriever_results
)

LLogan M

pip install llama-index-postprocessor-colbert-rerank

Plain Text

from llama_index.postprocessor.colbert_rerank import ColbertRerank

reranker = ColbertRerank(top_n=2)

...

retriever_results = reranker.postprocess_nodes(retriever_results)

...

jjma7889

got it, thank you

jjma7889

the reranker doesn't help. however, using sentence splitter and reducing the chunk to 384 made it work. chunk size seems critical. i still use the compact mode.
thank you for your help Logan, as always😀

Add a reply

Find answers from the community

Responw