Find answers from the community

Updated 2 months ago

Faiss

I'm storing my data in a FAISS vectorstore, and when I look at the docstore or index_store files, I see embeddings as null. Is it normal? Shouldnt they be stored in these files too?

Here is the way I generate the data:

for node in nodes:
        node_embedding = base_embeddings.get_text_embedding(
            node.get_content(metadata_mode="all")
        )
        node.embedding = node_embedding

vector_store.add(nodes)
index.storage_context.persist()

4 comments

LLogan M

This is Normal. Embeddings aren't recorded in the index store or docstore, only in the vector store (i.e. FAISS in this case)

BBaygon

ah great, thanks for the details. I'm using the metadata extractor as well, I guess this is only stored in the vector store too.... that would be interesting to see the metdata in cleartext though

LLogan M

Actually with faiss, it should be in the docstore

BBaygon

hmmm there is nothing.... I guess I'll rerun with DEBUG flag on to understand whats happening

Add a reply