Llamaindex Excluded Embed Metadata Keys

Question

Hi. Is there a way in llamaindex to restrict the metadata (keys) that should be stored in node? I want to exclude the unnecessary keys in the node and in the vector DB. I found this "document.excluded_embed_metadata_keys = ["key"]" setting being mentioned in one of the github issues, but it doesn't exclude the keys from being saved in the node. Thank you!

WhiteFang_Jr · Answer

Hey!When you add keys in document.excluded_embed_metadata_keys what it will do is exclude the following keys from embedding process. But it will stay as a metadata. If you want to remove some set of keys completely from the nodes, then once you get your docs/nodes you can can iterate over them and remove them before indexing happens

WhiteFang_Jr · Answer

docs = SimpleDirectorReader().load_data() for doc in docs: doc.metadata = {} #add the keys that you want to have in the nodes only or remove the ones from existing node # Then do the indexing part

MegaByte · Answer

I also thought about it but didn't go for it as I wanted to avoid the double implementation. Thank you once again for your quick response. 🙂

Find answers from the community

Llamaindex Excluded Embed Metadata Keys