Find answers from the community

Updated last year

hey, I'm using use_async to build index

At a glance
hey, I'm using use_async to build index from a lot of documents but it gives me Rate limit reached for text-embedding-ada-002 no matter how I adjust batch size. Changing to use_async = false fixes this but makes it very very slow and batch size dont apply when use_async = false for some reason. How can I embed a lot of documents quickly while without getting rate limit error? thank you!
T
T
3 comments
When using async false, how much volume is it doing? Do you get close to the OpenAI rate limits?
async = true: Generating embeddings: 45%| | 18/40 [01:11<01:27, 3.99s/it]
openai.error.RateLimitError: Rate limit reached for text-embedding-ada-002 in organization **** on tokens per min. Limit: 1000000 / min. Current: 992848 / min.
async = false: Generating embeddings: 88%|▏ | 17500/19999 [02:54<00:28, 86.19it/s], it stays /19999 no matter what batch size I set
Since you're hitting the OpenAI limit, you cannot really bypass that without requesting an increased quota
Add a reply
Sign up and join the conversation on Discord