Hi I’m having troubles with retriever

At a glance

Hi I’m having troubles with retriever 😭😭😭

Resulting documents are wrong. I tried hybrid search, but it didn’t improve the results.

Is there a way to send a list of keywords to maybe somehow navigate the retriever? I don’t even know what else to do please help 😭

@Logan M 🙏

25 comments

LLogan M

You tried cranking the top k and using reranking?

ppikachu8887867

@Logan M No, could you send a link where I can read about this?

LLogan M

You can use cohere or a local sentence-transformers model (I would recommend BAAI/bge-reranker-base these days for local models)
https://docs.llamaindex.ai/en/stable/module_guides/querying/node_postprocessors/node_postprocessors.html#coherererank

LLogan M

You can also use the LLM to rerank, but a little less stable

LLogan M

https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/rankGPT.html#retrieval-rankgpt-reranking

ppikachu8887867

Thanks! Let me try that out!

ppikachu8887867

Ah, so it will first run a normal retrieval, get the nodes, send them to LLM to verify If the nodes make sense, then use the verified nodes and send them to LLM?

ppikachu8887867

I'm not using any LLM in my pipeline, all I need is - find relevant nodes (pdf pages).

ppikachu8887867

@Logan M Basically, I don't need this:

Plain Text

response = query_engine.query(
    "Which grad schools did the author apply for and why?",
)

but this:

Plain Text

index = VectorStoreIndex.from_vector_store(vector_store=vector_store, service_context=service_context)

retriever = index.as_retriever(similarity_top_k=3)

nodes = retriever.retrieve(question)

LLogan M

yea you got it

ppikachu8887867

@Logan M Sorry, what model is being used here?

https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/SentenceTransformerRerank.html#:~:text=ctx%20%3D%20ServiceContext.from_defaults(embed_model%3D%22local%22)

ppikachu8887867

with nearly the same code I'm getting the error:

Plain Text

ValueError: shapes (13,768) and (384,) not aligned: 768 (dim 1) != 384 (dim 0)
~

LLogan M

bge-small-en-v1.5 I think

LLogan M

yea that makes sense lol

LLogan M

Need to use the same embedding model you already have -- which is fine, it's unrelated to reranking

ppikachu8887867

hmm.. so I need to use the BAAI/bge-reranker-base in service context?

Now I do this:

Plain Text

service_context = ServiceContext.from_defaults(embed_model=OpenAIEmbedding(model=OpenAIEmbeddingModelType.TEXT_EMBED_3_LARGE))

index = VectorStoreIndex.from_vector_store(vector_store=vector_store, 
service_context=service_context)


rerank = SentenceTransformerRerank(
        model="BAAI/bge-reranker-base",
        top_n=3
    )


query_engine = index.as_query_engine(similarity_top_k=10, 
node_postprocessors=[rerank])

return query_engine

LLogan M

Yea, the reranking isn't part of the service context

That could you have there makes sense

ppikachu8887867

@Logan M what do I need to change? xD

with the code above I'm getting:

Plain Text

ValueError: shapes (13,768) and (3072,) not aligned: 768 (dim 1) != 3072 (dim 0)

when I do:

Plain Text

nodes=query_engine.query(question)

LLogan M

uhhhh wtf lol

LLogan M

did you previously embed your data with openai 3 large?

ppikachu8887867

@Logan M I'm using qdrant with location = memory, so I create new embeddings every time I run the app. Regarding TEXT_EMBED_3_LARGE, yes, it did work until I've added rerank model 😄

ppikachu8887867

@Logan M Maybe that's because I'm creating my index with VectorStoreIndex.from_vector_store ? In the tutorial they are using: VectorStoreIndex.from_documents

LLogan M

nah that should be fine

ppikachu8887867

Okay I found the error xD

I declared IngestionPipeline above and I'm using fastembed there, but I declared OpenAIEmbeddings in service context 😄

Sorry bro 🙂

LLogan M

ah that makes sense!

Add a reply

Find answers from the community

Hi I’m having troubles with retriever