LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

hey guys just curious - reranker seems

RRZ
·
hey guys just curious - reranker seems to improve performance for my RAG, but makes it waaay longer to generate responses, any tips for reducing this?
L
R
5 comments
LLogan M
Use a reranker that doesn't run locally? (i.e. cohere?)
LLogan M
Or run your reranking on a GPU
RRZ
yeah makes sense
RRZ
just thinking maybe its possible to finetune a smaller reranker
RRZ
but maybe thats not great performance
Add a reply
Sign up and join the conversation on Discord