The community member is seeking advice or resources on "thresholding" the vectors retrieved from RAG (Retrieval Augmented Generation). They don't want to simply retrieve the top 10 most relevant vectors if only 3 are super relevant, or retrieve all 20 vectors if they are all crucial. Another community member suggests using a node_postprocessor for similarity during query engine initialization, setting top_k to 100 (or a preferred value) and a similarity threshold to classify correctly predicted nodes. They provide a link to the relevant documentation. The third community member acknowledges they will check this out.
Does anyone have any advice or experience or resources for ‘ thresholding’ the vectors you retrieve from RAG? What I mean is, I don’t want to just retrieve the top 10 most relevant and stuff/map-reduce whatever on all of them, if only 3 are super relevant. Likewise, maybe 20 vectors are all super crucial, in which case I want all of them
I think you can do this adding node_postprocessor for similarity at the time of query_engine initialization. Set top_k to 100( or your preferred value) and set similarity threshold value as what you think should be the benchmark for classifying as correctly predicted nodes.
