I'm running into errors with some of the

oosiworx

I'm running into errors with some of the embedding models

sentence-transformers/all-distilroberta-v1
it states it has 768 dimensions and then during generating the embeddings I get this error:
../aten/src/ATen/native/cuda/Indexing.cu:1290: indexSelectLargeIndex: block: [59,0,0], thread: [64,0,0] Assertion srcIndex < srcSelectDimSize failed.
it happens with all those embedding models stating to have 768 Dimensions I have tested so far 384, 512 dimensions was no issue and the embedding worked fine.

I run tests which embedding model would be best for our data and so I came across this issue. Do I do something wrong or is 768 Dimensions not working at all?

8 comments

LLogan M

How are you setting up the embed model? Do you have the latest version of llama-index-embeddings-huggingface installed?

oosiworx

I did pip install it last week

oosiworx

I do it like this embed_model = HuggingFaceEmbedding(model_name=embedding_models[model]['path'])

then when it comes to the embedding part after the parsing it crashes with that error

oosiworx

I got this version llama-index-embeddings-huggingface==0.1.4

LLogan M

Latest is v0.2.0

LLogan M

Switched to using the sentence-transformers library as the backend, instead of pure pytorch

LLogan M

Seems to work fine

Plain Text

>>> from llama_index.embeddings.huggingface import HuggingFaceEmbedding
>>> embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-distilroberta-v1")
>>> embeds = embed_model.get_text_embedding("Test")
>>> embeds = embed_model.get_query_embedding("Test")
>>>

oosiworx

ok I will try that

Add a reply

Find answers from the community

I'm running into errors with some of the