Find answers from the community

Updated 2 months ago

Node embeddings not visible in VectorStoreIndex

I want to look into VectorStoreIndex for the nodes and their embeddings and I have tried this snippet,
Plain Text
nodes = index.vector_store.get_nodes()
for node in nodes:
    print(f"Node ID: {node.node_id}, Embedding: {node.embedding}, Metadata: {node.metadata}")

But I am getting Embedding: None in all of them.
Is there any way I can see the generated embeddings of my nodes? And also I can't add huge amount of documents in SimpleDocumentStore using this
Plain Text
docstore = SimpleDocumentStore()
docstore.add_documents(nodes)

It gives me memory error after running for some time.
L
2 comments
Tru using the underlying vector store client (i.e the qdrant client, chroma client, etc.) -- embeddings are not returned in most cases to save memory/speed
SimpleDocumentStore is in memory, so yes, a huge amount of documents will crash your program if you run out of RAM -- there are many integrations that use mongodb, redis, etc, for the document store
Add a reply
Sign up and join the conversation on Discord