How to Deploy a Reranker on a GPU and Call That as a Service?

Question

how to deploy a reranker on a gpu and call that as a service?

Logan M · Answer

pretty sure text-embeddings-inference supports rerankers nowhttps://github.com/huggingface/text-embeddings-inference?tab=readme-ov-file#sequence-classification-and-re-ranking

Bhavya Giri · Answer

how do i call this from llamaindex

Find answers from the community

How to Deploy a Reranker on a GPU and Call That as a Service?