If I'm using `Settings.llm=Ollama(model

At a glance

The community member is using Settings.llm=Ollama(model="mistral") for their LLM and is wondering if they need to use a specific embedding model when creating a VectorStoreIndex from documents. They were previously using Settings.embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2").

The comments indicate that the community members can use any embedding model, as the embeddings are only used to look up the documents/nodes, and the actual text content from the document is what is passed as input/context to the LLM. The comments suggest that the token embeddings used by the LLM do not need to match the embeddings used for the VectorStoreIndex.

�🌿 ^( O v O )^🌿

If I'm using Settings.llm=Ollama(model="mistral") for my LLM, is there a specific embedding model I need to use when I'm trying to make a VectorStoreIndex from the documents? I was using HuggingFace Settings.embed_model = HuggingFaceEmbedding( model_name="sentence-transformers/all-MiniLM-L6-v2") ... does that make sense?

�

6 comments

TTeemu

Yeah you can use any embedding model

�🌿 ^( O v O )^🌿

So it doesn't matter if the LLM model was trained on different tokens? How does it know what the token embeddings "mean" if it's a different embedding model than the one the LLM was trained on?

TTeemu

Only the retrieved text is passed to the LLM

�🌿 ^( O v O )^🌿

ohhhh I see, so the embeddings are only being used to look up the Documents/Nodes, and then the actual text content from the document is what's being used as input/context for LLM?

TTeemu

Yeah basically

�🌿 ^( O v O )^🌿

ok gotcha - thanks

Add a reply

Find answers from the community

If I'm using `Settings.llm=Ollama(model