Find answers from the community

Updated 4 months ago

Cohere

At a glance

I have a question regarding Cohere Reranking based on the following - https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/CohereRerank.html

If Cohere is set as a node postprocessor as shown here, and there's a Cohere API outage (or say we have reached a rate limit or something like that), will this code still execute without any re-ranking, or will it crash?

4 comments

LLogan M

Hmm, it would probably raise a traceback about some network error I'd guess

SSayan

Cool, I can probably handle that as an exception and recreate the chat engine without cohere as a postprocessor. Thanks!

SSayan

@Logan M at what stage would it actually call the Cohere API?

Is it the as_query_engine() / as_chat_engine() , or the query() / chat() that comes after it?

LLogan M

During both.

Typically there is a a "retrieve" step, and reranking happens directly after that

Add a reply