Find answers from the community

Updated last year

Bm25

Hi Team, I successfully integrated a custom llm using the documentation guide. It works well and as expected with the default querying strategy. However, now I need to enhance retrieval and was looking to implement Hybrid search using BM25 and semantic search your guide from the docs. However, it always asks for an Open AI key (not for embedding). Could not load open ai model etc…
Can someone throw in a few suggestions? Thanks.

12 comments

LLogan M

How are you using bm25?

There's a few ways to set it up

LLogan M

You can pass in an index, a docstore, or even nodes themselves

LLogan M

Attachment

GGoku

Thanks for the reply @Logan M I just followed this doc here:
https://docs.llamaindex.ai/en/stable/examples/retrievers/reciprocal_rerank_fusion.html

It complains about no llm detected. My code implements the Custom LLM setup from the guide "Example: Using a Custom LLM Model - Advance".

It works perfectly find when using naive RAG with the custom llm, but when adding im Reciprocal Re-rank as shown in the docs, it gives the "No llm found error".

GGoku

I tried to dig through the library code and found that it errors out trying to find out if it's an instance on an llm in fusion_retriever.py

GGoku

And then I digged into one level deeper in the utils.py file and it errors out here.

Attachment

GGoku

Not sure why it enters line 18 when it shouldn't because my llm is not default. It's a custom llm.

GGoku

Doesn't seem like it's handling or checking if it's an instance of the CustomLLM class.

LLogan M

That function handles custom LLM just fine -- that function doesn't have to do anything if it's already a LLM object (it's just checking for other things, like langchain LLMs, disabling, etc.)

LLogan M

Just need to pass your LLM into the query fusion retirever

LLogan M

Plain Text

retriever = QueryFusionRetriever(..., llm=my_custom_llm)

GGoku

Thankyou for taking the time to respond, really appreciate it.

That's exactly what I did 🙂 but it still errors out and says it needs the openAI key. I'll review my code again and see I may have done wrong that's making it ignore my llm and post here if I find something.

Add a reply