The community member is looking for an API version of the SentenceTransformerRerank model, as using it directly in their code is resource-intensive. They are considering moving the rerank model to a separate API, so that their postprocessor can call the external API instead of running the model on their own server. In the comments, other community members suggest using API-based rerankers like Cohere Rerank or Jina, and note that implementing a node postprocessor that makes an API request is relatively straightforward. The community member acknowledges this and mentions they may need to self-host the API due to data residency requirements, and plans to create their own API-based reranker.
Hello, I was wondering is there an API version of SentenceTransformerRerank. When I use SentenceTransformerRerank directly in my code it takes up a lot of CPU / Ram. I'm thinking of moving the Rerank model into a separate API and so in the postprocessor, instead of running the rerank model on my my current code/server, it calls an external API to do so.
