Find answers from the community

Updated 7 months ago

llama_index/llama-index-integrations/vec...

Need help figuring out why Pinecone VectorIndexAutoRetriever is not working.


#### Background
I am working on a project using the LlamaIndex package integrating with Pinecone for my index. The project involves creating a VectorIndexAutoRetriever to set up a retriever similar to the example in https://docs.llamaindex.ai/en/stable/examples/vector_stores/pinecone_auto_retriever/

#### Problem Description
During the implementation, I encountered a ValueError stating that the vector store only supports exact match filters. Th error message is:
Plain Text
ValueError: Vector Store only supports exact match filters. Please use ExactMatchFilter or FilterOperator.EQ instead.

This error occurs when I attempt to use filters with operators other than exact matches (e.g., FilterOperator.LTE).

#### Investigation
  1. Filter Setup: The LLM gives the right translation from query to metadata filter. The query is Who are some founders that are 30 or below
    Plain Text
    MetadataFilters(filters=[MetadataFilter(key='age', value=30, operator=<FilterOperator.LTE: '<='>)], condition=<FilterCondition.AND: 'and'>)

    Pinecone also supports this operation '<=' as described by the code below:
    Plain Text
    def _transform_pinecone_filter_operator(operator: str) -> str:
        """Translate standard metadata filter operator to Pinecone specific spec."""
        print(f"DEBUG: The incoming operator is {operator} from the code")
        ...
        elif operator == ">=":
            return "$gte"
        elif operator == "<=":
            return "$lte"
        ...

    GitHub link
What am I doing wrong? Is only exact match filtering supported? I can help add documentation but currently its pretty confusing to debug further for me.
L
N
5 comments
Do you have the full traceback?
Some more useful logs
Plain Text
Vector store index created and ready for use.
DEBUGGG NEHIL [MetadataInfo(name='name', type='str', description='Name of the founder in the profile. Usually in headers'), MetadataInfo(name='profile_url', type='str', description='The url of the profile which is the url of the page'), MetadataInfo(name='linkedin_url', type='str', description='A linked in profile url attached to the profile. Could be None but very rarely.'), MetadataInfo(name='hobbies', type='list[str]', description='List of hobbies of the founder. ex: Cycling, Reading, Gaming'), MetadataInfo(name='employement_industries', type='list[str]', description='List of industries the founder has worked in. ex: AI, Real Estate, Travel'), MetadataInfo(name='location', type='str', description='Location of the founder. Usually in the format location: city, state, country. ex: location: New York, NY, USA'), MetadataInfo(name='age', type='int', description='Age of the founder. Usually in the format age: x. ex: age: 25')]
DEBUGING str_or_query_bundle in _retrieve Who are some founders that are 30 or below
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using query str: Profiles of founders from startup school YC platform
Using query str: Profiles of founders from startup school YC platform
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using filters: [('age', '<=', 30)]
Using filters: [('age', '<=', 30)]
Using query str: Profiles of founders from startup school YC platform
Using filters: [('age', '<=', 30)]
INFO:llama_index.core.indices.vector_store.retrievers.auto_retriever.auto_retriever:Using top_k: 2
Using top_k: 2
DEBUGING str_or_query_bundle in _retrieve Profiles of founders from startup school YC platform
@Logan M I was able to figure it out. I was not passing in storage context properly while creating the index. It was giving an error in an incorrect spot. So there might be a bug in where the error is caught.
Ah yes good catch. Following the traceback, we can see it's using the simple vector store, rather than your pinecone vector store πŸ‘
Add a reply
Sign up and join the conversation on Discord