Find answers from the community

Updated last year

Is it possible to load a JSON based

Is it possible to load a JSON-based vector index from storage (docstore.json, vector_store.json, etc.) and somehow get a list of all the nodes and their embeddings (call it all_nodes) so that all_nodes can be added to another type of index? Like this: vector_store.add(all_nodes_from_json) where vector_store is an OpenSearchVectorStore?
W
L
r
4 comments
I'm not sure, Every VectorStore Index works differently.
With the base vector index, since the embeddings are stored separately from the nodes, this is a little tricky

But, there is a way that is slightly hacky
Plain Text
# load the index
index = load_index_from_storage(...)

# get the nodes and embeddings
nodes = index.docstore.docs
embeddings = index.vector_store._data.embedding_dict

# attach the embeddings
nodes_with_embeddings = []
for node_id, node in nodes.items():
  node.embedding = embeddings[node_id]
  nodes_with_embeddings.append(node)

# create a new index with the new backend
vector_index = VectorStoreIndex(nodes_with_embeddings, storage_context=storage_context)
Thank you @Logan M ! That did the trick!
Add a reply
Sign up and join the conversation on Discord