Node embeddings not visible in VectorStoreIndex

I want to look into VectorStoreIndex for the nodes and their embeddings and I have tried this snippet,

Plain Text

nodes = index.vector_store.get_nodes()
for node in nodes:
    print(f"Node ID: {node.node_id}, Embedding: {node.embedding}, Metadata: {node.metadata}")

But I am getting Embedding: None in all of them.
Is there any way I can see the generated embeddings of my nodes? And also I can't add huge amount of documents in SimpleDocumentStore using this

Plain Text

docstore = SimpleDocumentStore()
docstore.add_documents(nodes)

It gives me memory error after running for some time.

2 comments

LLogan M

Tru using the underlying vector store client (i.e the qdrant client, chroma client, etc.) -- embeddings are not returned in most cases to save memory/speed

LLogan M

SimpleDocumentStore is in memory, so yes, a huge amount of documents will crash your program if you run out of RAM -- there are many integrations that use mongodb, redis, etc, for the document store

Add a reply

Find answers from the community

Node embeddings not visible in VectorStoreIndex