Hello I m getting some weird

Hello, I'm getting some weird AssertionError when using FAISS vector store. Any idea?
Adding the code in the threads

10 comments

I am first indexing my documents using this:

node_parser = SentenceWindowNodeParser.from_defaults(
    window_size=10,
    window_metadata_key="window",
    original_text_metadata_key="original_text",
)


llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
ctx = ServiceContext.from_defaults(
    llm=llm,
    embed_model=HuggingFaceEmbeddings(
        model_name="sentence-transformers/all-mpnet-base-v2"
    ),
    node_parser=node_parser,
)


"""
Setting up FAISS Vector Store
 - d = 768 to match the mpnet_base_v2 model
"""
d = 768
faiss_index = faiss.IndexFlatL2(d)
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

sentence_index = VectorStoreIndex.from_documents(
    all_docs, service_context=ctx, storage_context=storage_context, show_progress=True
)
sentence_index.storage_context.persist()

JJAX

I am loading my documents from the vector store using this code:

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
ctx = ServiceContext.from_defaults(
    llm=llm,
    embed_model=HuggingFaceEmbeddings(
        model_name="sentence-transformers/all-mpnet-base-v2"
    ),
    node_parser=node_parser,
)

vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
    vector_store=vector_store, persist_dir="./storage"
)
index = load_index_from_storage(storage_context=storage_context)


qa_template = Prompt(PROMPT)
query_engine = index.as_query_engine(
    similarity_top_k=5,
    text_qa_template=qa_template,
    # the target key defaults to

window

 to match the node_parser's default
    node_postprocessors=[
        MetadataReplacementPostProcessor(target_metadata_key="window")
    ],
    streaming=True,
)

JJAX

But when I have to ask the question, i'm getting this:

Traceback (most recent call last):
File "/mnt/d/Dev/poetry_llamaWindow/QueryFAISS.py", line 75, in <module>
window_response = query_engine.query(query)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/indices/query/base.py", line 23, in query
response = self._query(str_or_query_bundle)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/query_engine/retriever_query_engine.py", line 169, in _query
nodes = self.retrieve(query_bundle)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/query_engine/retriever_query_engine.py", line 117, in retrieve
nodes = self._retriever.retrieve(query_bundle)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/indices/base_retriever.py", line 22, in retrieve
return self._retrieve(str_or_query_bundle)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/indices/vector_store/retrievers/retriever.py", line 75, in _retrieve
return self._get_nodes_with_embeddings(query_bundle)
File "/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/llama_index/indices/vector_store/retrievers/retriever.py", line 151, in _get_nodes_with_embeddings
query_result = self._vector_store.query(query, **self._kwargs)
"/home/jax/.cache/pypoetry/virtualenvs/poetry-llamawindow-mJZojT2l-py3.10/lib/python3.10/site-packages/faiss/init.py", line 308, in replacement_search
assert d == self.d
AssertionError

JJAX

Any clue what this is about? It does seem related to FAISS vector store after googling it a bit, but I can find only some old references from back in the GPT Index days

LLogan M

Hmmm took a peek at faiss source code

Did you create the 8ndex with the same embedding model that you are querying with?

I thiiiiiiink it seems to be a dimension mismatch 🤔

JJAX

hey @Logan M - know that mpnet base 2 has 768 dimension and that's the one that i've set here:

d = 768
faiss_index = faiss.IndexFlatL2(d)

JJAX

Attachment

JJAX

model
SentenceTransformer(
(0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
(2): Normalize()
)

LLogan M

How are you setting the service context? Should make sure to pass it in when you load from disk too

JJAX

@Logan M , you, this seems to have fixed it, adding the context in load from index. Thanks again 🙏

Add a reply

Find answers from the community

Hello I m getting some weird