I'm using RetrieverEvaluator and getting

I'm using RetrieverEvaluator and getting different numbers each time I run it.

For example, the mrr might be 0.486083 the first time, but 0.473583 the second time.

Here's the retriever I'm evaluating:

Plain Text

    vector_store = get_chroma_vector_store(path)
    embed_model = resolve_embed_model("local:BAAI/bge-small-en-v1.5")
    service_context = ServiceContext.from_defaults(llm=None, embed_model=embed_model)
    index = VectorStoreIndex.from_vector_store(
        vector_store,
        service_context=service_context,
    )
    retriever = index.as_retriever(similarity_top_k=5)

Find answers from the community

I'm using RetrieverEvaluator and getting