Find answers from the community

Updated 10 months ago

Hey Folks

At a glance
Hey Folks
I tried to Opensearch Vectorstore demo.

Facing issue on the parsing the vectors. and tried to use BAAI/bge-small-en-v1.5 model embedding instead of openai

https://docs.llamaindex.ai/en/stable/examples/vector_stores/OpensearchDemo/

Plain Text
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings

# define embedding function
Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-small-en-v1.5"
) 
text_field = "content"
# OpensearchVectorClient stores embeddings in this field by default
embedding_field = "embedding"
# OpensearchVectorClient encapsulates logic for a
# single opensearch index with vector search enabled
client = OpensearchVectorClient(
    endpoint, idx, dim=384, embedding_field=embedding_field, text_field=text_field
)
# initialize vector store
vector_store = OpensearchVectorStore(client)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# initialize an index using our sample data and the client we just created
index = VectorStoreIndex.from_documents(
    documents=documents, storage_context=storage_context
)



facing a issue
Plain Text
BulkIndexError: ('22 document(s) failed to index.', [{'index': {'_index': 'gpt-index-demo', '_id': '941f4570-ac78-431b-b88c-287d9882aa46', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': "failed to parse field [embedding] of type [knn_vector] in document with id '941f4570-ac78-431b-b88c-287d9882aa46'. Preview of field's value: 'null'", 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Vector dimension mismatch. Expected: 1536, Given: 384'}}


Can anyone pls advice me on this πŸ˜ƒ
d
S
3 comments
you need to speciy the embedding models in your settings
oh wait you did lol try passing the embedd_model into the index

embedd_model =HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5"
)
Settings.embed_model =embed_model
Hey @Duh_Kola
It works when specifying embedd_model and passing to the Settings.embed_model.
not sure what happened to be honest
But thanks for this help πŸ˜ƒ
Add a reply
Sign up and join the conversation on Discord