Find answers from the community

Updated 4 months ago

Openai

At a glance

Hi all. Does llama-index handle the OpenAI API retry in query engine for RateLimitError? I see some code in the repo openai_utils.py to handle the retry. But I also see llama-index uses lang chain openai api wrapper which seems throwing RateLimitError out. Should I handle it in my code?

4 comments

LLogan M

Where are we using an openai wrapper from langchain? Could have sworn all our LLM code was in house

The current retry logic is a little basic right now, I think it might retry too quickly to properly handle the rate limit error

LLogan M

Would definitely appreciate a PR to make that better though

jjma7889

I didn't copied down the last ratelimit error I encounter a few days ago. I think it was similar to this one https://github.com/jerryjliu/llama_index/blob/main/docs/examples/vector_stores/LanceDBIndexDemo.ipynb

jjma7889

while i am not sure openai_utils.py is used in query engine, i feel it is only used in direct openai query call.

Add a reply