Find answers from the community

Updated 7 months ago

Is there some kind of migration needed

Is there some kind of migration needed for upgrading llama-index-embeddings-huggingface from 0.1.4 to 0.2.1
I used to use model and tokenizer as parameters in the HuggingFaceEmbedding. After updating to 0.2.1 I need to install sentencepiece and it's saying model and tokenizer is deprecated.

4 comments

LLogan M

it moved to using sentence transformers under the hood, rather than raw pytorch and huggingface

provide the model name/path and it should be fine

cchantlong

Seems like the sentence transformer has to be less than 3.0.0 but greater than 2.6. I shouldn't need to install sentencepiece right due to the dependency requirement of embeddings-huggingface 0.2.1

cchantlong

nvm I guess it really depends on the huggingface model used.

LLogan M

Yea different models will require sentencepiece for the tokenizer

Add a reply