Hi!
I'm having an issue with the milvus vector store. The delete function contains this code:
entries = self._milvusclient.query(
collection_name=self.collection_name,
filter=f"{self.doc_id_field} in [{','.join(doc_ids)}]",
)
if len(entries) > 0:
ids = [entry["id"] for entry in entries]
self._milvusclient.delete(collection_name=self.collection_name, pks=ids)
logger.debug(f"Successfully deleted embedding with doc_id: {doc_ids}")
The problem is that the first line can fail with
grpc: received message larger than max
. I have documents with over 1,000 chunks, so I often run into this even when deleting a single document.
Is there a reason why this code is written this way instead of this:
res = self._milvusclient.delete(
collection_name=self.collection_name,
filter=f"{self.doc_id_field} in [{','.join(doc_ids)}]",
)
logger.debug(f"Successfully deleted {res.get('delete_count', 0)} nodes from the vector store")
?