Find answers from the community

Updated last month

Evaluating a Hybrid Search Functionality for a Specific Use Case

I have build a search functionality over my dataset and built a hybrid retriever which takes BM25Retriever and VectorIndexRetriever into account. I want to now evaluate this searching mechanism. All evaluations are being done with question and answer pairs right now and do not fit into my case. What are possible solutions of evaluating this in my case? Any suggestions are welcome.
L
3 comments
You could evaluate based on similarity to the ground truth
Or some measure of overlap, like ROUGE
Or if you don't have ground truth, you basically need an LLM to evaluate the retrieval
Add a reply
Sign up and join the conversation on Discord