Find answers from the community

Updated 3 months ago

Bm25

Hi, while trying to use bm25_retriever = BM25Retriever.from_defaults(
docstore=index.docstore, similarity_top_k=2
), getting this error - ZeroDivisionError
I am using weaviate as vector store and the loaded index does have a good amount of documents, How to resolve this error?

5 comments

LLogan M

Bm25 requires a docstore or nodes.

When using weaviate (or other vector db integrations) the docstore is disabled by default to simplify storage

Try passing in nodes directly to the bm25 receiver instead?

PPujari12

is it possible to pass all the nodes of an index?

LLogan M

Only if you still have the nodes from when you created the index. Or maybe do a retrieve with the top k set to a large number lol

PPujari12

Hey, which vector DB to use for doing hybrid or fusion search? I need something around which I can build my own customised ranking

LLogan M

I did just release sparse embedding support for qdrant (which is a type of hybrid search)
https://docs.llamaindex.ai/en/stable/examples/vector_stores/qdrant_hybrid.html

You can implement your own reranking function for when you fuse sparse and dense results

Add a reply