LlamaIndex
Log in

Find answers from the community

Updated last year

hello @Logan M . Got a question,

JJAX
·
hello @Logan M . Got a question, hopefully you can help me clarify my confusion. I've recently read the documentation where using OpenAI Embeddings + Reranking (BGE / Cohere) should significantly increase the hit rate on retrieval. Currently I am using the all-mpnet-base-v2 embedding with the NodeSentenceWindow retrieval. I am considering to switch to OpenAI Embeddings using BGE Reranker Base (mainly due to the fact that our documentation changes frequently and to recalculate a ton of embeddings every week does not really scale well). I am confused a little bit about how to make this type of retrieval to work with reranking.
J
L
5 comments
JJAX
My understanding is this:
  • first i retrieve my top K documents based on the "sentence embeddings"
  • I pass the associated window to the LLM to compile an answer
If i am to use reranking, then it will be something like:
  • i retrieve the top k documents based on sentence embeddings
  • before passing to the LLM, i need to rerank the documents but here is the question - do I rerank them based on the window or based on the sentence ?
  • pass them to LLM for answer
LLogan M
It depends on the order you apply the post-processors. But my gut says re-ranking on the entire window makes more sense
JJAX
Thanks for the reply! I assume I need to do a custom function that basically takes the retrieved nodes from the base retriever and then do reranking on the window text from all nodes. The current implementation does not do that by default, no? 😄
JJAX
when you say it depends, you mean in the query engine when i provide the list of postprocessors, if i do rerank first then metadata preprocessor or the other way around? i would assume metadata postprocessor and then reranking but i dont know if i can set the reranker to look at the window metadata key instead of text key
LLogan M
If you apply the metadata replacement processor first (i.e. its first in the list), then any subsequent post-processors will oeprate on the window
Add a reply
Sign up and join the conversation on Discord