Filters

why is my MetatadataFilter not working as I'd expect?

Plain Text

filters = MetadataFilters(filters=[
        ExactMatchFilter(
            key="is_published",
            value=1
        ),
    ])

    index = get_redis_index()
    retriever = index.as_retriever(filters=filters)
    response = retriever.retrieve('how can I deal with my credit card?')
    if not response:
        print('No documents found')
    for doc in response:
        print(doc.metadata)

output:

Plain Text

(groove) (base) ➜  blobby-server git:(master) ✗ python -m src.redis_index
17:54:26 redisvl.index.index INFO   Index already exists, not overwriting.
No documents found

versus:

Plain Text

filters = MetadataFilters(filters=[
        ExactMatchFilter(
            key="is_published",
            value=0
        ),
    ])

output:

Plain Text

groove) (base) ➜  blobby-server git:(master) ✗ python -m src.redis_index
17:55:06 redisvl.index.index INFO   Index already exists, not overwriting.
{'content_type': 'text', 'content_id': 'rboPV4SXTMdVcXoZ', 'page_type': 'article', 'slug': 'what-can-i-do-with-my-401-k-after-i-leave-my-job', 'title': 'What can I do with my 401k after I leave my job?', 'keywords': 'what to do 401k leaving job', 'description': 'If you recently switched jobs or retired, you might be wondering what to do with your old 401(k) plan. You have four options with a 401(k) when leaving your job or retiring.', 'subtitle': '', 'author': 'Noa Rodriguez-Hoffman', 'is_published': 1}

This seems backwards to me...

2 comments

LLogan M

That's kind of sus. How are you building the index?

DDustfurn

@Logan M I agree! Here's my implementation of get_redis_index:

Plain Text

def get_redis_index(
    namespace: str = "blobby", redis_host: Optional[str] = None, redis_port: Optional[str] = None,
    redis_pwd: Optional[str] = None
) -> VectorStoreIndex:
    if not redis_host:
        redis_host = os.getenv('REDIS_HOST')
    if not redis_port:
        redis_port = os.getenv('REDIS_PORT')
    if not redis_pwd:
        redis_pwd = os.getenv('REDIS_PASSWORD')

    redis_client = redis.Redis(host=redis_host, port=redis_port, password=redis_pwd)
    vector_store = RedisVectorStore(
        schema=get_index_schema(namespace=namespace),
        redis_client=redis_client,
    )
    embed_model = OpenAIEmbedding(model="text-embedding-3-large")
    return VectorStoreIndex.from_vector_store(vector_store, embed_model=embed_model)

Add a reply

Find answers from the community

Filters