I am running into an issue with trying

ggarnizzle

I am running into an issue with trying to add nodes to a qdrant vector store. Apparently the nodes need to already be embedded but I cannot find where in the source code or in the documentation it says how to use llama index to embed the nodes. Here is my code - can someone please help me fix this? Its probably a one-liner I am missing. It is failing at index.vector_store.add(nodes)

Plain Text

def create_engine_with_nodes(self, nodes):
        index = self.create_index()
        index.vector_store.add(nodes)
        
        model = OpenAI(model=self.LLM)
        query_engine = self.create_engine_from_index(index, model)
        return query_engine

    def create_engine(self):
        index = self.create_index()
        model = OpenAI(model=self.LLM)
        query_engine = self.create_engine_from_index(index, model)
        return query_engine
    
    def create_index(self):
        embedding_model = OpenAIEmbedding(model=OpenAIEmbeddingModelType.TEXT_EMBED_3_LARGE, dimensions=1024)

        if self.USE_QDRANT:
            client = qdrant_client.QdrantClient(
                os.environ.get("QDRANT_CLOUD_ENDPOINT"),
                api_key=os.environ.get("QDRANT_API_KEY"),
                grpc_port=6334, 
                prefer_grpc=True,
                timeout=30
            )
            
            print("Creating Qdrant Vector Store Index from nodes")
            vector_store = QdrantVectorStore(client=client, collection_name=self.collection_name, parallel=5)
            index = VectorStoreIndex.from_vector_store(vector_store=vector_store, embed_model=embedding_model)       
        else:
            index = VectorStoreIndex(embed_model=embedding_model)

        return index

29 comments

ggarnizzle

I see that there is a method, get_text_embedding_batch, but if I use this then the embeddings are disconnected from the nodes and there is no easy way to put them back together.

LLogan M

The embeddings are the same order as the the inputs were given, so you can reattach them (this is what llamaindex does for you)

index.insert_nodes(nodes) will do all the embedding for you as well

ggarnizzle

Thanks for getting back to me. Is ordering guaranteed?

LLogan M

yes

LLogan M

otherwise creating a VectorStoreIndex object wouldn't work 😅

ggarnizzle

When I try that, I get this error:

Plain Text

Exception has occurred: TypeError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
create() got an unexpected keyword argument 'dimensions'

I am setting up the embedding model based on what I read in the source code and the documentation - it appears that dimensions is a valid argument for the constructor:
self.embedding_model = OpenAIEmbedding(model=OpenAIEmbeddingModelType.TEXT_EMBED_3_LARGE, dimensions=1024)

I also get the same error trying to do response = query_engine.query(query)

Any idea how to fix this?

ggarnizzle

Also if I try something like this that error triggers:

Plain Text

pipeline = IngestionPipeline(
            transformations=[
                SentenceSplitter(),
                self.embedding_model,
            ]
        )

LLogan M

It is indeed a valid argumen. It seems like you are missing the full traceback, but if I had to guess, its related to either your openai version or llama-index-embeddings-openai version?

This worked for me

Plain Text

>>> embed_model = OpenAIEmbedding(model="text-embedding-3-large", dimensions=1024)
>>> embed = embed_model.get_text_embedding("Hello world!")
>>> len(embed)
1024
>>>

Plain Text

openai==1.14.3
llama-index-embeddings-openai==0.1.6

ggarnizzle

My package versions

Plain Text

llama-index-core==0.10.26
llama-index-embeddings-openai==0.1.7
llama-index-llms-openai==0.1.14
llama-index-vector-stores-qdrant==0.2.8
llamaindex-py-client==0.1.15
openai==1.5.0

ggarnizzle

Maybe a few are old, I can update everything now

LLogan M

I think openai especially will probably help fix it (that seems like where the error was coming from)

ggarnizzle

testing it now. Most of those libraries were installed yesterday but core and openai are older so I just updated them

ggarnizzle

ok now I am getting another weird error:

Plain Text

objc[79306]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[79306]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[79307]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[79307]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[79309]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[79309]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[79310]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[79310]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
objc[79308]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.

this might be related to my attempt to parallelize it but I am not sure:
vector_store = QdrantVectorStore(client=client, collection_name=self.collection_name, parallel=5)

LLogan M

hmmm, thats a new one for me. I've never used that param before though.

ggarnizzle

removing that parameter fixed the ingestion, now that part works. unfortunately we are going to ingest a lot of data so doing it in parallel would be nice

LLogan M

if you also pass in an async client QdrantVectorStore(..., client=QdrantCleint(..), aclient=AsyncQdrantClient(...))

Then you can do async/concurrant inserts (and probably a little safer than parallel) await vector_store.async_add(nodes)

ggarnizzle

ok thats a good tip!

ggarnizzle

it appears to ingest properly now, but I cannot retrieve. this line of code returns an empty response:

response = query_engine.query(query)

but if I check the results of just the retrieveal, it does work.

Plain Text

retriever = query_engine.retriever
 results = retriever.retrieve(query)
print(len(results)) # prints 10

something is going wrong where it fails to synthesize the response. How can I fix this?

LLogan M

Are you using a specific LLM?

LLogan M

And how did you construct the query engine?

ggarnizzle

Plain Text

self.LLM = "gpt-4-1106-preview"


model = OpenAI(model=self.LLM)
query_engine = index.as_query_engine(
            similarity_top_k=self.TOP_K,
            node_postprocessors=[
                SimilarityPostprocessor(similarity_cutoff=self.SIMILARITY_SCORE_CUTOFF),
                SentenceTransformerRerank(model="cross-encoder/ms-marco-MiniLM-L-2-v2", top_n=self.RERANK_CUTOFF)
            ],
            llm=model,
            response_mode=ResponseMode.SIMPLE_SUMMARIZE
        )

ggarnizzle

we are using a preview version of GPT4, it seems to have the best results. I can try to swap it

ggarnizzle

still fails swapping the LLM to 'gpt-3.5-turbo' and 'gpt-4'

LLogan M

your similarity cutoff is probably removing all the nodes

LLogan M

if I had to guess

ggarnizzle

that fixed it. I had to remove the cutoff completely. Its interesting that it was still returning 10 results from the retriever but the synthesizer was where the it actually did the cutoff

ggarnizzle

thanks for your help

LLogan M

You can think of it as the query engine being a wrapper around a bunch of indepdendant modules

Without the query engine, you would have to

run the retriever
run each of your postprocessors
using the remaining nodes, invoke the synthesizer

(And you can actually do all these steps yourself if you wanted)

Add a reply

Find answers from the community

I am running into an issue with trying