Find answers from the community

Updated last month

The double retries in the openai llm

At a glance

The community member noticed that the OpenAI LLM seems to have double retries, where the max_retries parameter is passed to both the OpenAI client and the llm_retry_decorator, leading to retrying 3x3 times. The community members discussed this issue, with one noting that the OpenAI client is still wrapped with a retry decorator, and another stating they are unsure about the logic behind OpenAI's retries. The community members considered making a pull request to address the issue, but no definitive solution or answer was provided.

Useful resources
Hello. I noticed that the OpenAI LLM seems to have double retries: the max_retries parameter is passed to openai's OpenAI client ("native" SDK) as well as it is used by llm_retry_decorator which wraps methods such as chat. This leads to retrying 3x3 times when we think we only retry 3 times. Is that on purpose or am I missing something?
p
L
4 comments
Does this really fix the issue? OpenAI client is still wrapped with a retry decorator, but the OpenAI is again initialized with retries: https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/embeddings/llama-index-embeddings-openai/llama_index/embeddings/openai/base.py#L347
Regarding llm_retry_dectorator vs OpenAI's logic, I don't know. Haven't explored OpenAI's logic for retries
yea true. Idk, might make a PR if I have time. Or if you want to go for iy
Thanks for your answers. Will try to make a PR in the next days/weeks if I find time for this
Add a reply
Sign up and join the conversation on Discord