Find answers from the community

Updated 3 months ago

Metadata

Hey Team,
We are adding a custom retriever for search, where we require a metadata field (array of string) added under the metadata but while doing so we received the following error [Attached Screenshot: Llama-index insertion.png]. But when we tried with pinecone core package, we are able to successfully upsert the document [Attached Screenshot: Pinecone insertion.png].

I also remember this is already being discusesd by earlier [Sorry, Couldn't find the ref to exact thread]
Any estim. here how long it might gonna take for this functionality to be added in llama-index for pinecone?

/ /
Attachments
Pinecone_insertion.png
Llama-index_insertion.png
2
W
L
r
7 comments
Hi!

Did you try inserting the metadata like this?
some vector databases require that the keys must be strings, and the values must be flat (either str, float, or int)

Plain Text
from llama_index import SimpleDirectoryReader
filename_fn = lambda filename: {'file_name': filename}

# automatically sets the metadata of each document according to filename_fn
documents = SimpleDirectoryReader('./data', file_metadata=filename_fn).load_data()

For ref:
https://docs.llamaindex.ai/en/stable/core_modules/data_modules/documents_and_nodes/usage_documents.html#customizing-documents
Yea I think for this, the pinecone vector store is (incorrectly?) Requiring flat metadata

Would be a simple PR to fix, just flipping a boolean attribute. Assuming that it's the correct thing to do
Yeah, making default value flat_metadata = False - https://github.com/run-llama/llama_index/blob/main/llama_index/vector_stores/pinecone.py . Should solve the issue.
@ravitheja if you verified that this works, feel free to open a PR for it!
Yeah working on it.
Thanks team for quick help, Much appreciated πŸ™
Add a reply
Sign up and join the conversation on Discord