Hi all, I am using bge-small embedding

rrachel_001.

Hi all, I am using bge-small embedding model to do the embeddings for my documents, and want to use PGVectorStore to store the vectors (embed_dim=384). The challenge i am having now is that: DataError: (psycopg2.errors.DataException) expected 1536 dimensions, not 384 .

18 comments

LLogan M

You set PGVectorStore(..., embed_dim=384) ?

Did you query your vector store with the same embedding model that you used to index it?

Any code sample you have might help

rrachel_001.

Plain Text

 vector_store = PGVectorStore.from_params(
    database=default_db,
    host=pg_host,
    password=pg_password,
    port=pg_port,
    user=pg_user,
    table_name="test_chunk_512",
    embed_dim=384,  # embedding dimension for openai is 1536
)

rrachel_001.

here are my codes

Plain Text

 index = VectorStoreIndex.from_documents(
    documents, 
    storage_context=storage_context, 
    #service_context=service_context,
    embed_dim=384,
    show_progress=True
)

rrachel_001.

yes I set embed_dim as 384. my codes are paseted above.

LLogan M

Plain Text

 index = VectorStoreIndex.from_documents(
    documents, 
    storage_context=storage_context, 
    #service_context=service_context,
    embed_dim=384,
    show_progress=True
)

It doesn't look like you configured the embedding model? Normally you would see

Plain Text

 index = VectorStoreIndex.from_documents(
    documents, 
    storage_context=storage_context, 
    embed_model=embed_model,
    show_progress=True
)

rrachel_001.

Still got the same error after i configure the the embed_model in the VectorStoreIndex function.

Plain Text

DataError: (psycopg2.errors.DataException) expected 1536 dimensions, not 384

LLogan M

Mmm not possible lol

Share the fully current code again? What line is causing the error?

rrachel_001.

Sharing the code

Plain Text

index = VectorStoreIndex.from_documents(
    documents, 
    storage_context=storage_context, 
    #service_context=service_context,
    embed_dim=384,
    embed_model=embed_model,
    show_progress=True
)

LLogan M

Is this the line of code causing the error?

And how is the storage context setup?

rrachel_001.

yes this line is causing the error. here are code for storage context:

Plain Text

vector_store = PGVectorStore.from_params(
    database=default_db,
    host=pg_host,
    password=pg_password,
    port=pg_port,
    user=pg_user,
    table_name="test_chunk_512",
    embed_dim=384,  # embedding dimension for bge-small is 384
)

# create index
storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = VectorStoreIndex.from_documents(
    documents, 
    storage_context=storage_context, 
    #service_context=service_context,
    embed_dim=384,
    embed_model=embed_model,
    show_progress=True
)

rrachel_001.

Sharing the codes

rrachel_001.

@Logan M and here is the part of the error

Plain Text

DataException                             Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/sqlalchemy/engine/base.py:2120, in Connection._exec_insertmany_context(self, dialect, context)
   2119     else:
-> 2120         dialect.do_execute(
   2121             cursor,
   2122             sub_stmt,
   2123             sub_params,
   2124             context,
   2125         )
   2127 except BaseException as e:

File /opt/conda/lib/python3.10/site-packages/sqlalchemy/engine/default.py:919, in DefaultDialect.do_execute(self, cursor, statement, parameters, context)
    918 def do_execute(self, cursor, statement, parameters, context=None):
--> 919     cursor.execute(statement, parameters)

DataException: expected 1536 dimensions, not 384


The above exception was the direct cause of the following exception:

DataError                                 Traceback (most recent call last)
Cell In[26], line 1
----> 1 index = VectorStoreIndex.from_documents(
      2     documents, 
      3     storage_context=storage_context, 
      4     #service_context=service_context,
      5     embed_dim=384,
      6     embed_model=embed_model,
      7     show_progress=True
      8 )

LLogan M

Is this with a brand new table?

rrachel_001.

No worries, the above error has been fixed =). Thank you. However, i am now having another issue with HuggingFaceLLM function. I run the code below but got an valueerror " ValueError: You are trying to offload the whole model to the disk. Please use the disk_offload function instead." Not pretty where to configure the disk_offload

Plain Text

llm = HuggingFaceLLM(
    model_name="meta-llama/Meta-Llama-3-8B",
    model_kwargs={
        "token": token_hf,
        "torch_dtype": torch.bfloat16,  # comment this line and uncomment below to use 4bit
        # "quantization_config": quantization_config
        
    },
    generate_kwargs={
        "do_sample": True,
        "temperature": 0.01,
        "top_p": 0.9,
    },
    tokenizer_name="meta-llama/Meta-Llama-3-8B",
    tokenizer_kwargs={"token": token_hf},
    stopping_ids=stopping_ids,
    
)

LLogan M

This means you don't have enough memory (either VRAM and/or RAM) to run the model. If you actually offload to disk, it will be unsuably slow

LLogan M

Try using ollama instead imo, much easier for local dev

rrachel_001.

Well honestly I am running this code in SageMaker studio notebook. Is ollama still better than huggingface when deploying into SageMaker?

LLogan M

hmm not sure. But sounds like you just need a bigger instance to deploy on then

Add a reply

Find answers from the community

Hi all, I am using bge-small embedding