Find answers from the community

Updated 6 months ago

Batch

At a glance

The community members are discussing the possibility of using LlamaIndex to perform batch text embeddings, similar to the 24-hour turnaround embeddings provided by OpenAI. One community member notes that this feature is not currently available in LlamaIndex, but they are open to contributions. Another community member suggests providing guidelines on how to check if the embedding request has been processed, and proposes a potential UX design for submitting and retrieving the embeddings, including handling large batch sizes and paged responses. However, there is no explicitly marked answer in the comments.

ttheoxd

is there a way using llama index to do embeddings using batchs requests ? (the openai ones that have 24 hours turnover?)

5 comments

LLogan M

Not at the moment. Been meaning to add something for that but tbh it's been low priority

LLogan M

Open to contributions though

ttheoxd

if you have general guide lines (for ex what you would like to see on how to check if the request is processed) i'd be happy to do so

LLogan M

I think some UX on the embedding model that allows for

submitting the job (i.e. job_name = embed_model.submit_text_embeddings(texts))
checking if the results are ready, and if so, returning the embeddings (batch jobs are generally huge, I wonder if this needs to be an iterator? I'm not sure how openai gives you the embeddings from the response, probably a paged response?) -- Maybe something like for embedding_batch in embed_model.get_embedding_job_results(job_name):

LLogan M

Just quick thoughts without looking too deep lol

Add a reply