Find answers from the community

Updated 2 months ago

Responw

I am using retriever with top k = 10 and a generic responseSynthesizer, i can see the result is in node 5, however no matter what mode I use (I tried all of them), the result is always wrong (n/a mostly, and a couple of wrong answers). But, if I use a keyword post processor to narrow the results to 1 node, then the same responseSynthesizer returns the correct answer. I suspect if I ask LLM with each node, it will have one correct answer. Not sure why llama index synthesizer mostly gives N/A.
What should I do to improve the responseSynthesizer quality?
L
j
7 comments
You can try a different response mode ("compact" is default, there's also "refine" and "tree_summarize")

You could also use a reranking step to narrow down the top k
i tried tree_summarize, i tried all of them listed
Probably then I would use a reranker. 10 nodes is a lot of text to get lost in for most LLMs
Plain Text
retriever = VectorIndexRetriever(
    index=index, similarity_top_k=10, response_mode="tree_summarize"
)

retriever_results = retriever.retrieve(
    CRITERION_PERIOD_OF_PERFORMANCE_LLM_RETRIEVE_PROMPT
)
response_synthesizer = get_response_synthesizer(
    structured_answer_filtering=True
)

response_synthesizer_results = response_synthesizer.synthesize(
    CRITERION_PERIOD_OF_PERFORMANCE_LLM_QUESTION, retriever_results
)
pip install llama-index-postprocessor-colbert-rerank

Plain Text
from llama_index.postprocessor.colbert_rerank import ColbertRerank

reranker = ColbertRerank(top_n=2)

...

retriever_results = reranker.postprocess_nodes(retriever_results)

...
got it, thank you
the reranker doesn't help. however, using sentence splitter and reducing the chunk to 384 made it work. chunk size seems critical. i still use the compact mode.
thank you for your help Logan, as always😀
Add a reply
Sign up and join the conversation on Discord