shapes not aligned error

tthoraxe

probably user error. I am using the HF TEI server to embed, but then when I try to do a lookup, I get this error:
ValueError: shapes (1024,) and (768,) not aligned: 1024 (dim 0) != 768 (dim 0)
https://gist.github.com/thoraxe/583ee9f8d2a21a562f42535da47cee0d

36 comments

tthoraxe

I made sure to nuke the local db/index and reindex before trying

tthoraxe

it's definitely hitting the embedding server during the querying

LLogan M

hmmm you are extra sure you blew away the old index at vector-db/ocp-product-docs ?

tthoraxe

i will be triple extra sure one sec

tthoraxe

Plain Text

⌁68% [thoraxe:~/Red_Hat/ … ft/llamaindex-experiments/fastapi-lightspeed-service] [fastapi-ols-39] fix-rag(+12/-1)* 1 ± ls vector-db/
ocp-product-docs  summary-docs
⌁68% [thoraxe:~/Red_Hat/ … ft/llamaindex-experiments/fastapi-lightspeed-service] [fastapi-ols-39] fix-rag(+12/-1)* ± rm -rf vector-db/*
⌁68% [thoraxe:~/Red_Hat/ … ft/llamaindex-experiments/fastapi-lightspeed-service] [fastapi-ols-39] fix-rag(+12/-1)* ± ls vector-db/
⌁68% [thoraxe:~/Red_Hat/ … ft/llamaindex-experiments/fastapi-lightspeed-service] [fastapi-ols-39] fix-rag(+12/-1)* ±

tthoraxe

same error

tthoraxe

ValueError: shapes (1024,) and (768,) not aligned: 1024 (dim 0) != 768 (dim 0)

tthoraxe

let me crank up debug logging

tthoraxe

hmm, i didn't get any more logging details

LLogan M

And you are sure the service context/embed model is the same when indexing vs. querying?

LLogan M

maybe add a print(service_context.embed_model) before performing each step to confirm

tthoraxe

it should be, the code is the same

tthoraxe

but i will check

tthoraxe

2023-11-10 12:26:33,955 [docs_summarizer.py:76] INFO: 1234 using embed model: model_name='BAAI/bge-base-en-v1.5' embed_batch_size=10 callback_manager=<llama_index.callbacks.base.CallbackManager object at 0x7f6290d58e20> base_url='xxx' query_instruction=None text_instruction=None timeout=60.0 truncate_text=True

tthoraxe

nvm

tthoraxe

the model name is the same for both when I index and when I query

tthoraxe

no, wait, it's not.

tthoraxe

something weird is going on

tthoraxe

AHHHHH

tthoraxe

poop.

tthoraxe

user error, I knew it.

tthoraxe

one sec

tthoraxe

yeah ok, so in reality while I thought I was using the embedding server when I was indexing, I was in fact not because that file was not loading env from dotenv

tthoraxe

and the shell i happened to run the test from didn't have the TEI server set, so it was actually embedding locally

LLogan M

oooo

tthoraxe

i just assumed it was still working since yesterday

tthoraxe

but this si good, now i print the embed model 🙂

LLogan M

Nice, good catch!

tthoraxe

well, you caught it