Hi, I would like to test the effect of varying different parameters in my RAG app (embed model, reranker, etc...) on the retrieval performance. Specifically, I would like to input a dictionary of pairs of user queries and associated correct node ids (where the answer to the query can be found), and then compute the % of times a set of parameters finds the correct node in top place. Does llamaindex offer an easy way to do such benchmarking? I haven’t seen that in the doc, but I prefer to be sure before coding this from scratch. 🙂 thanks.
