I am building an index with 300k

moreover, an ideas for optimization the running time? i use model_name=BAAI/bge-small-en can it run on GPU?

About optimization

https://gpt-index.readthedocs.io/en/latest/examples/llm/llama_2_llama_cpp.html#installation

Thanks

ops

this one as well

i don't find in the docs how to improve the embeddings speed

i am using the local embedding model - Could not load OpenAIEmbedding. Using HuggingFaceBgeEmbeddings with model_name=BAAI/bge-small-en. If you intended to use OpenAI, please check your OPENAI_API_KEY.

i mean, how can I run the BAAI/bge-small-en faster? what if I add a GPU to my mahcine

maybe @Logan M can jump in here

You can try with a better hardware, but the embedding it self doesn't have other solutions to improve speed, mainly this Langchain one, with OpenAIEmbeddings you would increase the batch size

GPU will definitely help here

I think if you have cuda installed, it will automatically run on gpu too

oh nice, i'll try on cuda gpu

Yo @Bar Haim

Maybe it can interest you as well

https://x.com/jerryjliu0/status/1706097203486629889?s=20