Find answers from the community

Updated 2 months ago

i am setting up my rag evaluation pipeline, here is my code ```python import osfrom d

i am setting up my rag evaluation pipeline,
here is my code

Plain Text
python 
import os
from dotenv import load_dotenv
import nest_asyncio

load_dotenv()
nest_asyncio.apply()

from llama_index.core.evaluation import RetrieverEvaluator

Settings.embed_model = OpenAIEmbedding(model="text-embedding-ada-002", embed_batch_size=10)
llm = OpenAI(model="gpt-4o")

client = qdrant_client.QdrantClient(
    url=os.getenv("QDRANT_URI"), api_key=os.getenv("QDRANT_API_KEY")
)
vector_store = QdrantVectorStore(client=client, collection_name="mlofo-loan-officer-july")
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

qa_dataset = EmbeddingQAFinetuneDataset.from_json("pg_eval_dataset.json")


metrics = ["mrr", "hit_rate"]

retriever_evaluator = RetrieverEvaluator.from_metric_names(
    metrics, retriever=index.as_retriever(similarity_top_k=2)
)

sample_id, sample_query = list(qa_dataset.queries.items())[0]
sample_expected = qa_dataset.relevant_docs[sample_id]

eval_result = retriever_evaluator.evaluate(sample_query, sample_expected)
print(eval_result)

generate the dataset
Plain Text
nodes = vector_store.get_nodes()

qa_dataset = generate_question_context_pairs(
    nodes, llm=llm, num_questions_per_chunk=2
)


reference: https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/cookbooks/cohere_retriever_eval.ipynb

error

File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 184, in _aget_nodes_with_embeddings
query_result = await self._vector_store.aquery(query, **self._kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/vector_stores/qdrant/base.py", line 927, in aquery
response = await self._aclient.search(
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'search'
k
p
L
6 comments
Please stand by @payload as I search the knowledge base. I normally take about 20 seconds or so. :wave:
----------------------
The error message you're seeing, AttributeError: 'NoneType' object has no attribute 'search', suggests that the search method is being called on a NoneType object. In your case, it seems like self._aclient is None.

This could be due to a problem with the initialization of your QdrantClient. You're initializing it with the following line:

Plain Text
client = qdrant_client.QdrantClient(
    url=os.getenv("QDRANT_URI"), api_key=os.getenv("QDRANT_API_KEY")
)


Make sure that the environment variables QDRANT_URI and QDRANT_API_KEY are correctly set and accessible in your environment. You can check this by printing their values before initializing the QdrantClient:

Plain Text
print(os.getenv("QDRANT_URI"))
print(os.getenv("QDRANT_API_KEY"))


If these print None, it means these environment variables are not set correctly.

Also, ensure that the Qdrant server is running and accessible from your application. The QDRANT_URI should be the URL where your Qdrant server is running.

If you're still facing issues, it might be helpful to refer to the LlamaIndex documentation () or the source code () for more examples and information.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/evaluation/retrieval/evaluator.py", line 56, in _aget_retrieved_ids_and_texts
retrieved_nodes = await self.retriever.aretrieve(query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 255, in async_wrapper
result = await func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/base/base_retriever.py", line 274, in aretrieve nodes = await self._aretrieve(query_bundle=query_bundle) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 255, in async_wrapper result = await func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 112, in _aretrieve return await self._aget_nodes_with_embeddings( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 184, in _aget_nodes_with_embeddings query_result = await self._vector_store.aquery(query, self._kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/payload/miniconda3/envs/mloflo/lib/python3.12/site-packages/llama_index/vector_stores/qdrant/base.py", line 927, in aquery
response = await self._aclient.search(
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'search'

@Logan M @WhiteFang_Jr need help here
You did not define the async qdrant client
QdrantVectorStore(..., client=QdrantClient(...), aclient=AsyncQdrantClient(...))
thanks i missed that we have to initiate async client separately
Add a reply
Sign up and join the conversation on Discord