Find answers from the community

Updated 2 months ago

Why specified hf embeddings do not work for indexing and retrieval

Can someone offer any insight why my specified HF embeddings do not work ? I use one .py file for indexing with the following content (relevant to the question) and the same pgvector DB for retrieving the data.
App part:
Plain Text
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")
vector_store = PGVectorStore.from_params(
    database=db_name,
    host=url.host,
    password=db_password,
    port=url.port,
    user=db_user,
    table_name=table_name,
    embed_dim=1024,  # HF embedding dimension
)
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

Separate indexer py :
Plain Text
vector_store_connection_string = f"postgresql://{db_user}:{db_password}@{db_host}:{db_port}/{db_name}"
url = make_url(vector_store_connection_string)

vector_store = PGVectorStore.from_params(
    database=db_name,
    host=url.host,
    password=db_password,
    port=url.port,
    user=db_user,
    table_name=table_name,
    embed_dim=1024,  # HF embedding dimension
)

# Create the storage context and index
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, show_progress=True
)

print("Indexing complete. Data stored in PGVector.")

The error :
Plain Text
DataError: (psycopg2.errors.DataException) different vector dimensions 1536 and 1024 [SQL: SELECT public.data_building_permits_data.id, public.data_building_permits_data.node_id

Any insight is appreciated.
W
L
17 comments
In your separate indexer file you do not have defined the embedding model so it is picking OpenAI( as its the default )
Try defining it in that file as well:
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")
Sorry for not specifying. I do it in both files...
i mixed the order....
Plain Text
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")
Settings.llm = Ollama(model="llama3.1:70b", base_url="http://<domain>.si:11434", request_timeout=60, context_window=6092)
this is from app
I am deploying everything on a Openshift Cluster (Docker images). The app worked fine when i had pgvector and code running locally. I also do not understand where this part of OpenAI embeddings comes from. I do have a "Failover mechanism" written like that in my main app :
Plain Text
# Function to check if the local server is available
def is_local_server_available(url):
    try:
        response = httpx.get(url, timeout=5.0)
        return response.status_code == 200
    except httpx.RequestError:
        return False
    
# Check if the local Ollama server is available
ollama_base_url = "http://>domain>.si:11434"
if is_local_server_available(ollama_base_url):
    Settings.llm = Ollama(model="llama3.1:70b", base_url=ollama_base_url, timeout=60.0, temperature=0.0,)
else:
    openai_api_key = os.getenv("OPENAI_API_KEY")
    Settings.llm = OpenAI(api_key=openai_api_key, model="gpt-4", temperature=0)
For some reason your embedding model is not being loaded
and since you have the openai key, it is creating openai model for emebdding
understood. I will remove this and debug further.
Can you print and check before creating index?
print(Settings.embed_model)
Check this just before creating index and see which model is there
Will do... the issue you pointed seems correct.
I will notify you in this thread if its okay.
Thank you for fast response
The issue was specified older build image (in my deployment.yaml). Everything works fine now.
Add a reply
Sign up and join the conversation on Discord