Find answers from the community

Updated 3 months ago

Hello, I was wondering is there an API

Hello, I was wondering is there an API version of SentenceTransformerRerank.
When I use SentenceTransformerRerank directly in my code it takes up a lot of CPU / Ram.
I'm thinking of moving the Rerank model into a separate API and so in the postprocessor, instead of running the rerank model on my my current code/server, it calls an external API to do so.
L
c
2 comments
I mean, theres cohere rerank, jina, a few other api-based rerankers.

A node postprocessor is a single method, so its very easy to implement one that just makes an api request
i see thanks. I'll prob. need to self host one bc of data residency requirements. Will make my own then haha
Add a reply
Sign up and join the conversation on Discord