ARRAY_CONTAINS on MilvusVectorStore

HHazmat1027

Hey, y'all! I'm trying to filter my Milvus collection by an array and I'm having some trouble. There doesn't seem to be a way to use Milvus's ARRAY_CONTAINS method in the MetadataFilter class. Has anyone solved for this problem yet?

19 comments

HHazmat1027

For context, I'm using a MilvusVectorStore. Creating a MetadataFilter on a string-valued property works:

Plain Text

expr = {
            "key": "id",
            "operator": FilterOperator.IN,
            "value": "xyz"
        }

metadata_filter = MetadataFilter.from_dict(expr)
metadata_filters = MetadataFilters(filters=[metadata_filter])

vsq = VectorStoreQuery(
        query_embedding=[0.0] * 3072,
        similarity_top_k=top_k,
        filters=metadata_filters
    )

collection.query(query=query)

If I try the same thing with an array or a value contained in an array, I can't get the filter to work:

Plain Text

expr = {
            "key": "codes",
            "operator": FilterOperator.EQ,
            "value": "code1"
        }

expr = {
            "key": "codes",
            "operator": FilterOperator.EQ,
            "value": ["code1","code2"]
        }

HHazmat1027

I'm thinking that the VectorStore class is the wrong thing for what I'm trying to do

LLogan M

Probably the vector store integration needs to be updated to support this specific filter

HHazmat1027

Yeah, it says that it only supports the EQ operator for this specific function. I was hoping someone knew of a workaround.

HHazmat1027

I can use the pymilvus MilvusClient to filter fairly easily in a test scenario, but the llama_index VectorStore is what I'm actually using in my app. Bummer.

LLogan M

It would be a fairly easy PR for someone who knows how to use milvus 🙂

HHazmat1027

So I made some updates in a forked repo but I'm not seeing them when I run my Jupyter notebook. I ran pip install e . and restarted my Jupyter kernel as described in the contribution guide but am not seeing any updates. Am I missing something?

HHazmat1027

@Logan M

LLogan M

when you say you ran pip install -e ., what directory were you in when you ran that?

Generally, I would have done pip install -e llama-index-integrations/vector-stores/llama-index-vector-stores-milvus/ from the root dir

HHazmat1027

I've done both

HHazmat1027

I actually jsut missed a line in init.py

HHazmat1027

All good!

LLogan M

nice!

HHazmat1027

I have a PR mostly ready. I'm kinda lost on the tool.llamahub part of the commit message. It looks like these are all the same. What exactly am I supposed to be editing?

HHazmat1027

i.e.

Plain Text

[tool.llamahub]
contains_example = false
import_path = "llama_index.vector_stores.milvus"

[tool.llamahub.class_authors]
MilvusVectorStore = "llama-index"

They're all like this.

LLogan M

Its just used to map

the main import path for an integration
the modules it exposes
who authored those modules (i.e. the github username, llama-index is just default)

LLogan M

You don't have to edit this usually

HHazmat1027

Oh, okay. Reading it a little closer, it's only in the context of a new package being added. My bad.

LLogan M

No worries!

Add a reply

Find answers from the community

ARRAY_CONTAINS on MilvusVectorStore