Find answers from the community

Updated 5 months ago

Trying to understand how I can run an

At a glance
Trying to understand how I can run an embedding model locally on my computer to avoid sending sensitive data to cloud server.

In the documentation, HuggingFaceEmbedding is the class that’s used for local embeddings. Is that taking care of downloading the model from HuggingFace or do I have to make a manual installation of the model first?
L
t
8 comments
It will download the model for you 👍
Thanks! Where will the model be stored?
by default, in /tmp/llama_index -- but you can configure this by setting LLAMA_INDEX_CACHE_DIR in your env
When using a local embedding model, I have now understood that I have to define how many dimensions the models are using. What I dont get is /where/ in my code I do this. The way I understand the docs, it should be when configuring the StorageContext before loading the index. But StorageContext doesn't accept as an argument. So that isn't it.
Attachment
CleanShot_2023-12-07_at_21.42.042x.png
You only have to define this when you are using a vector store that requires this. Otherwise you can ignore it
The main requirement is that you use the same embedding model for queries as you used for building the index
I'm not using any addon vector store yet, just following the Start Tutorial. Is that what's causing me trouble when switching to local embedding model?
Attachment
CleanShot_2023-12-07_at_22.22.372x.png
Yea if you switch embedding models, you need to re-create the index

And when you load an index, make sure you pass the service context again

load_index_from_storage(storage_context, service_context=service_context)
Add a reply
Sign up and join the conversation on Discord