Find answers from the community

Updated 6 months ago

Hi!

Hi!

I'm having an issue with the milvus vector store. The delete function contains this code:
Plain Text
        entries = self._milvusclient.query(
            collection_name=self.collection_name,
            filter=f"{self.doc_id_field} in [{','.join(doc_ids)}]",
        )
        if len(entries) > 0:
            ids = [entry["id"] for entry in entries]
         self._milvusclient.delete(collection_name=self.collection_name, pks=ids)
            logger.debug(f"Successfully deleted embedding with doc_id: {doc_ids}")

The problem is that the first line can fail with grpc: received message larger than max . I have documents with over 1,000 chunks, so I often run into this even when deleting a single document.

Is there a reason why this code is written this way instead of this:
Plain Text
    res = self._milvusclient.delete(
        collection_name=self.collection_name,
        filter=f"{self.doc_id_field} in [{','.join(doc_ids)}]",
    )
    logger.debug(f"Successfully deleted {res.get('delete_count', 0)} nodes from the vector store")

?
L
M
3 comments
Maybe filtered deletes weren't implemented yet when the delete method was written?
Definitely make a PR with the change if you are up for it
Will look into that. Thanks
Add a reply
Sign up and join the conversation on Discord