LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

Indexing/Embedding question. How can I

eedk
·
Indexing/Embedding question. How can I speed up the embedding/indexing process of llama index. I have possibly thousands of documents I want to index as fast as possible. Any help is appreciated.
T
e
L
6 comments
TTeemu
Are you using the OpenAI API? You can increase the batch size https://gpt-index.readthedocs.io/en/latest/module_guides/models/embeddings.html
eedk
@Teemu much appreciated. It really helped. What's the most/fastest batch size I can request if you happen to know
TTeemu
It will depend on your personal rate limits, I probably have different ones so it's hard to say. You can try higher ones and see what happens
TTeemu
Also if I'm not mistaken, they recently upped the rate limits (at least mine). Not sure if the standard batch size reflects that change yet 🤔
eedk
Many thanks. By the way, do you know if I can multiprocess/distribute indexing of documents or is it inherently a synchronous process
LLogan M
I think the max batch size for OpenAI is 2048

you should be able set use_async=True in the index constructor to also help speed up requests
Add a reply
Sign up and join the conversation on Discord