Find answers from the community

Updated 2 months ago

Embedding large documents

i have close 1000 txt documents need to be indexed using vector db and whole process should take pretty long time. is there anyway i can know the % of index progress ?

1 comment

LLogan M

You can set the show_progress boolean
https://gpt-index.readthedocs.io/en/latest/core_modules/data_modules/index/usage_pattern.html#high-level-api

You can also try changing the batch size (the default is 10)

https://gpt-index.readthedocs.io/en/latest/core_modules/model_modules/embeddings/usage_pattern.html#batch-size

Add a reply