Find answers from the community

Updated 3 months ago

Metadata

hi guys, I'm trying to pass a list of entities as metadata along my inserted documents, but I'm getting back an error: Value for metadata entities must be one of (str, int, float, None)

21 comments

LLogan M

Are you using a vector db integration? Some vector dbs require metadata to be a flat dictionary :PSadge:

LLogan M

Quick fix is casting to a string or similar

vvedtam

I'm using Pinecone, but that suports list of stirngs

vvedtam

Attachment

vvedtam

tried converting the list to a set (flat dict) via: set(["entity.."])

vvedtam

which gave back {"entity"} but still getting the error

LLogan M

Sorry, when I said flat dict, I meant to cast the entire list or set as a string. I.e the entire metadata dictionary us being forced to str -> str,int,float

LLogan M

Seems like we need to allow a list of strings for pinecone at some point 🤷‍♂️

vvedtam

like this? str(entities)

vvedtam

sorry, I'm not excelling in python

vvedtam

especially when it comes to its type system 😛

LLogan M

No worries, and pretty much

Or if you are just updating an existing metadata dictionary, something like metadata["entities"] = str(metadata["entitites"])

vvedtam

ok, so when I'm inserting along new documents, I do:

Plain Text

entities = ["Tom", "Peter"]
metadata["entities"] = str(entities)
documents.append(Document(
        id_=str(s['id']),
        text=s.get('bodytext', ''),
        metadata=metadata
}

vvedtam

my concern is, that I won't be able to look up items within these converted arrays later via:

Plain Text

query_engine = index.as_query_engine(
    similarity_top_k=3,
    filters=MetadataFilters(
        filters=[
            ExactMatchFilter(key="user_id", value="12345")
        ]
    ),
)

vvedtam

for instance, how would I look for a particular entity if the list becomes a continuous string?

vvedtam

just checked

vvedtam

was hoping for some type coerson but no

vvedtam

data goes as string instead of list into pinecone

Attachment