Find answers from the community

Updated 2 months ago

Rate limits

Hi!, im getting 'Rate limit reached for 10KTPM-200RPM' errors when using gpt-4. Should LlamaIndex take into consideration the limit's, and sleep between calls? or thats something i'll need to do at the app side?

2 comments

LLogan M

We do have some basic retry logic, but the values (wait time and number of retries) are hard-coded.

Would be a nice PR to make these user configurable 😅

I think you'll have to manage it on the app side it looks like otherwise

kkittenkill

all right, thanks for the info!

Add a reply