Why specified hf embeddings do not work for indexing an...

LLoLiPoPMaN

Can someone offer any insight why my specified HF embeddings do not work ? I use one .py file for indexing with the following content (relevant to the question) and the same pgvector DB for retrieving the data.
App part:

Plain Text

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")
vector_store = PGVectorStore.from_params(
    database=db_name,
    host=url.host,
    password=db_password,
    port=url.port,
    user=db_user,
    table_name=table_name,
    embed_dim=1024,  # HF embedding dimension
)
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

Separate indexer py :

Plain Text

vector_store_connection_string = f"postgresql://{db_user}:{db_password}@{db_host}:{db_port}/{db_name}"
url = make_url(vector_store_connection_string)

vector_store = PGVectorStore.from_params(
    database=db_name,
    host=url.host,
    password=db_password,
    port=url.port,
    user=db_user,
    table_name=table_name,
    embed_dim=1024,  # HF embedding dimension
)

# Create the storage context and index
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, show_progress=True
)

print("Indexing complete. Data stored in PGVector.")

The error :

Plain Text

DataError: (psycopg2.errors.DataException) different vector dimensions 1536 and 1024 [SQL: SELECT public.data_building_permits_data.id, public.data_building_permits_data.node_id

Any insight is appreciated.

17 comments

WWhiteFang_Jr

In your separate indexer file you do not have defined the embedding model so it is picking OpenAI( as its the default )

WWhiteFang_Jr

Try defining it in that file as well:
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")

LLoLiPoPMaN

Sorry for not specifying. I do it in both files...

LLoLiPoPMaN

i mixed the order....

LLoLiPoPMaN

Plain Text

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-m3")
Settings.llm = Ollama(model="llama3.1:70b", base_url="http://<domain>.si:11434", request_timeout=60, context_window=6092)

LLoLiPoPMaN

this is from app

LLoLiPoPMaN

I am deploying everything on a Openshift Cluster (Docker images). The app worked fine when i had pgvector and code running locally. I also do not understand where this part of OpenAI embeddings comes from. I do have a "Failover mechanism" written like that in my main app :

Plain Text

# Function to check if the local server is available
def is_local_server_available(url):
    try:
        response = httpx.get(url, timeout=5.0)
        return response.status_code == 200
    except httpx.RequestError:
        return False
    
# Check if the local Ollama server is available
ollama_base_url = "http://>domain>.si:11434"
if is_local_server_available(ollama_base_url):
    Settings.llm = Ollama(model="llama3.1:70b", base_url=ollama_base_url, timeout=60.0, temperature=0.0,)
else:
    openai_api_key = os.getenv("OPENAI_API_KEY")
    Settings.llm = OpenAI(api_key=openai_api_key, model="gpt-4", temperature=0)

WWhiteFang_Jr

For some reason your embedding model is not being loaded

WWhiteFang_Jr

and since you have the openai key, it is creating openai model for emebdding

LLoLiPoPMaN

understood. I will remove this and debug further.

WWhiteFang_Jr

Can you print and check before creating index?
print(Settings.embed_model)
Check this just before creating index and see which model is there

LLoLiPoPMaN

Will do... the issue you pointed seems correct.

LLoLiPoPMaN

I will notify you in this thread if its okay.