llama_index/llama-index-integrations/vec...

Need help figuring out why Pinecone VectorIndexAutoRetriever is not working.

#### Background
I am working on a project using the LlamaIndex package integrating with Pinecone for my index. The project involves creating a VectorIndexAutoRetriever to set up a retriever similar to the example in https://docs.llamaindex.ai/en/stable/examples/vector_stores/pinecone_auto_retriever/

#### Problem Description
During the implementation, I encountered a ValueError stating that the vector store only supports exact match filters. Th error message is:

Plain Text

ValueError: Vector Store only supports exact match filters. Please use ExactMatchFilter or FilterOperator.EQ instead.

This error occurs when I attempt to use filters with operators other than exact matches (e.g., FilterOperator.LTE).

#### Investigation

Filter Setup: The LLM gives the right translation from query to metadata filter. The query is Who are some founders that are 30 or below

Plain Text

MetadataFilters(filters=[MetadataFilter(key='age', value=30, operator=<FilterOperator.LTE: '<='>)], condition=<FilterCondition.AND: 'and'>)

Pinecone also supports this operation '<=' as described by the code below:

Plain Text

def _transform_pinecone_filter_operator(operator: str) -> str:
    """Translate standard metadata filter operator to Pinecone specific spec."""
    print(f"DEBUG: The incoming operator is {operator} from the code")
    ...
    elif operator == ">=":
        return "$gte"
    elif operator == "<=":
        return "$lte"
    ...

GitHub link

What am I doing wrong? Is only exact match filtering supported? I can help add documentation but currently its pretty confusing to debug further for me.

5 comments

LLogan M

Do you have the full traceback?

NNehil

Yep 1 sec

NNehil

Some more useful logs

Plain Text

Vector store index created and ready for use.
DEBUGGG NEHIL [MetadataInfo(name='name', type='str', description='Name of the founder in the profile. Usually in headers'), MetadataInfo(name='profile_url', type='str', description='The url of the profile which is the url of the page'), MetadataInfo(name='linkedin_url', type='str', description='A linked in profile url attached to the profile. Could be None but very rarely.'), MetadataInfo(name='hobbies', type='list[str]', description='List of hobbies of the founder. ex: Cycling, Reading, Gaming'), MetadataInfo(name='employement_industries', type='list[str]', description='List of industries the founder has worked in. ex: AI, Real Estate, Travel'), MetadataInfo(name='location', type='str', description='Location of the founder. Usually in the format location: city, state, country. ex: location: New York, NY, USA'), MetadataInfo(name='age', type='int', description='Age of the founder. Usually in the format age: x. ex: age: 25')]
DEBUGING str_or_query_bundle in _retrieve Who are some founders that are 30 or below
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using query str: Profiles of founders from startup school YC platform
Using query str: Profiles of founders from startup school YC platform
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using filters: [('age', '<=', 30)]
Using filters: [('age', '<=', 30)]
Using query str: Profiles of founders from startup school YC platform
Using filters: [('age', '<=', 30)]
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using top_k: 2
Using top_k: 2
DEBUGING str_or_query_bundle in _retrieve Profiles of founders from startup school YC platform

NNehil

@Logan M I was able to figure it out. I was not passing in storage context properly while creating the index. It was giving an error in an incorrect spot. So there might be a bug in where the error is caught.

LLogan M

Ah yes good catch. Following the traceback, we can see it's using the simple vector store, rather than your pinecone vector store 👍

Add a reply

Find answers from the community

llama_index/llama-index-integrations/vec...

Need help figuring out why Pinecone VectorIndexAutoRetriever is not working.