Find answers from the community

Updated 3 months ago

Using this code to generate an index

Using this code to generate an index from scratch:
Plain Text
def nodes_to_faiss(nodes, persist_dir):

    # Generate Node Embeddings
    embed_model = OpenAIEmbedding(api_key=OPENAI_KEY)

    for node in nodes:
        node_embedding = embed_model.get_text_embedding(
            node.get_content(metadata_mode="all")
        )
        node.embedding = node_embedding

    # Build the Index from the Nodes
    d = 1536
    faiss_index = faiss.IndexFlatL2(d)
    vector_store = FaissVectorStore(faiss_index=faiss_index)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    index = VectorStoreIndex(
        nodes=nodes,
        storage_context=storage_context,
    )

    # save the resulting index to disk so that we can use it later
    print("Index created. Saving to disk...")
    index.storage_context.persist(persist_dir=persist_dir)

This used to work fine, but now I am getting this error:
Plain Text
RetryError: RetryError[<Future at 0x21da34162b0 state=finished raised InvalidRequestError>]
L
M
2 comments
I'm going to guess that one of your nodes has no text

I've seen the error before, replicated with embed_model.get_text_embedding("")
Ah I see, I didn't realize that could cause an error. Thanks so much!
Add a reply
Sign up and join the conversation on Discord