Find answers from the community

Updated 2 months ago

The double retries in the openai llm

At a glance

The community member noticed that the OpenAI LLM seems to have double retries, where the max_retries parameter is passed to both the OpenAI client and the llm_retry_decorator, leading to retrying 3x3 times. The community members discussed this issue, with one noting that the OpenAI client is still wrapped with a retry decorator, and another stating they are unsure about the logic behind OpenAI's retries. The community members considered making a pull request to address the issue, but no definitive solution or answer was provided.

Useful resources

ppaull

Hello. I noticed that the OpenAI LLM seems to have double retries: the max_retries parameter is passed to openai's OpenAI client ("native" SDK) as well as it is used by llm_retry_decorator which wraps methods such as chat. This leads to retrying 3x3 times when we think we only retry 3 times. Is that on purpose or am I missing something?

4 comments

ppaull

Does this really fix the issue? OpenAI client is still wrapped with a retry decorator, but the OpenAI is again initialized with retries: https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/embeddings/llama-index-embeddings-openai/llama_index/embeddings/openai/base.py#L347

ppaull

Regarding llm_retry_dectorator vs OpenAI's logic, I don't know. Haven't explored OpenAI's logic for retries

LLogan M

yea true. Idk, might make a PR if I have time. Or if you want to go for iy

ppaull

Thanks for your answers. Will try to make a PR in the next days/weeks if I find time for this

Add a reply