Find answers from the community

Updated 2 months ago

Evaluating a Hybrid Search Functionality for a Specific Use Case

I have build a search functionality over my dataset and built a hybrid retriever which takes BM25Retriever and VectorIndexRetriever into account. I want to now evaluate this searching mechanism. All evaluations are being done with question and answer pairs right now and do not fit into my case. What are possible solutions of evaluating this in my case? Any suggestions are welcome.

3 comments

LLogan M

You could evaluate based on similarity to the ground truth

LLogan M

Or some measure of overlap, like ROUGE

LLogan M

Or if you don't have ground truth, you basically need an LLM to evaluate the retrieval

Add a reply