Find answers from the community

Updated 9 months ago

Topic: nested `MetadataFilters`

Topic: nested MetadataFilters
I am attempting to use MetadataFilters (similar to how this article describes: https://docs.llamaindex.ai/en/stable/examples/multi_tenancy/multi_tenancy_rag.html)
Recently, nested MetadataFilters were introduced in version 0.10.19
https://github.com/run-llama/llama_index/pull/11778
which fixes:
***
Notably. this changes the code in llama_index.core.vector_stores.types.py::MetadataFilters:
Plain Text
class MetadataFilters(BaseModel):
    """Metadata filters for vector stores."""

    # Exact match filters and Advanced filters with operators like >, <, >=, <=, !=, etc.
    filters: List[Union[MetadataFilter, ExactMatchFilter, "MetadataFilters"]]

However, when I run the code below, I encounter the following errors
L
h
9 comments
I think nested metadata filters are only handled by the postgres vector store πŸ€”
But the error is happening prior to any attempts to persist the data to any vector store. Simply trying to run a query results in the exception
What is the error? I don't remeber if you posted it)
It's posted just below my original post (with more context)
Running r = engine.query("some question") yields

Plain Text
File /path/to/lib/python3.10/site-packages/llama_index/core/vector_stores/types.py:170, in MetadataFilters.legacy_filters(self)
    166 for filter in self.filters:
--> 170     if filter.operator != FilterOperator.EQ:
    171         raise ValueError(
    172             "Vector Store only supports exact match filters. "
    173             "Please use ExactMatchFilter or FilterOperator.EQ instead."
    174         )
    175     filters.append(ExactMatchFilter(key=filter.key, value=filter.value))

AttributeError: 'MetadataFilters' object has no attribute 'operator'
However I think you might be correct, VectorStoreIndex's default storage_context is SimpleDocumentStore and the PR is just for PGVectorStore

I thought it would be a more intuitive design if the engine's representation was decoupled from it's persistence-backend/storage-context

Since the MetadataFilters class is now defined as:
Plain Text
class MetadataFilters(BaseModel):
    filters: List[Union[MetadataFilter, MetadataFilters]]

one would assume that it just "works" for other storage-contexts... other ppl will run into this issue
@Logan M I'm just slowly beginning to go through the underlying llama-index codebase - can you confirm if nested metadata filters have to be implemented independently for every storage-context?!
they have to be implemented for each vector store yes. Since they all have their own filtering logic
seems like an issue here with using nested filters + the older legacy operation (not every vector db has moved off of legacy)
its a lot of work :PSadge:
Add a reply
Sign up and join the conversation on Discord