Dustfurn

Filters

why is my MetatadataFilter not working as I'd expect?

Plain Text

filters = MetadataFilters(filters=[
        ExactMatchFilter(
            key="is_published",
            value=1
        ),
    ])

    index = get_redis_index()
    retriever = index.as_retriever(filters=filters)
    response = retriever.retrieve('how can I deal with my credit card?')
    if not response:
        print('No documents found')
    for doc in response:
        print(doc.metadata)

output:

Plain Text

(groove) (base) ➜  blobby-server git:(master) ✗ python -m src.redis_index
17:54:26 redisvl.index.index INFO   Index already exists, not overwriting.
No documents found

versus:

Plain Text

filters = MetadataFilters(filters=[
        ExactMatchFilter(
            key="is_published",
            value=0
        ),
    ])

output:

Plain Text

groove) (base) ➜  blobby-server git:(master) ✗ python -m src.redis_index
17:55:06 redisvl.index.index INFO   Index already exists, not overwriting.
{'content_type': 'text', 'content_id': 'rboPV4SXTMdVcXoZ', 'page_type': 'article', 'slug': 'what-can-i-do-with-my-401-k-after-i-leave-my-job', 'title': 'What can I do with my 401k after I leave my job?', 'keywords': 'what to do 401k leaving job', 'description': 'If you recently switched jobs or retired, you might be wondering what to do with your old 401(k) plan. You have four options with a 401(k) when leaving your job or retiring.', 'subtitle': '', 'author': 'Noa Rodriguez-Hoffman', 'is_published': 1}

This seems backwards to me...

2 comments

DDustfurn

I have prototyped a retrieval system

I have prototyped a retrieval system using the basic SimpleStore classes. I have persisted the data to disk. Now, I want to copy all that data into corresponding RedisStores. How would I do this?

For example, Here are my two storage contexts:

Simple:

Plain Text

StorageContext(docstore=<llama_index.core.storage.docstore.simple_docstore.SimpleDocumentStore object at 0x1497fd570>, index_store=<llama_index.core.storage.index_store.simple_index_store.SimpleIndexStore object at 0x1497fde40>, vector_stores={'image': <llama_index.core.vector_stores.simple.SimpleVectorStore object at 0x1497fe890>, 'default': <llama_index.core.vector_stores.simple.SimpleVectorStore object at 0x1497fe9e0>}, graph_store=<llama_index.core.graph_stores.simple.SimpleGraphStore object at 0x1497fde70>)

Redis:

Plain Text

StorageContext(docstore=<llama_index.storage.docstore.redis.base.RedisDocumentStore object at 0x1497fe4d0>, index_store=<llama_index.storage.index_store.redis.base.RedisIndexStore object at 0x1497fe110>, vector_stores={'default': RedisVectorStore(stores_text=True, is_embedding_query=True, stores_node=True, flat_metadata=False), 'image': <llama_index.core.vector_stores.simple.SimpleVectorStore object at 0x1497fe5c0>}, graph_store=<llama_index.core.graph_stores.simple.SimpleGraphStore object at 0x1497fe4a0>)

25 comments

DDustfurn

How do i persist and re create a

How do i persist and re-create a CondenseQuestionChatEngine ?

In particular, my use case is that I'd like to allow a user to have multiple chat instances that they can resume at any time. Ideally, an instance of a chat engine could be serliazed (to disk, a db, or some other form of storage), and deserialized at a later date.

5 comments

DDustfurn

Question 1:

Question 1:

In a VectorStore, what is the difference between a doc_id and a ref_doc_id (all mine are the same value)?

Question 2:

Given a doc_id or ref_doc_id, how can i get all the nodes from a VectorStore with that id?

I tried:

vector_store.query(VectorStoreQuery(doc_ids=[my_doc_id]))

but i get :

ValueError: Query embedding is required for querying.

1 comment

Find answers from the community

Filters

I have prototyped a retrieval system

How do i persist and re create a

Question 1: