Find answers from the community

Updated 9 months ago

Hello, I have a simple question! is the

Hello, I have a simple question! is the meta data excluded by default in the embeddings generation and LLM?
this is my code :

Plain Text

from llama_index.core import SimpleDirectoryReader

filename_fn = lambda filename: {"file_name": filename}

# automatically sets the metadata of each document according to filename_fn
documents = SimpleDirectoryReader(
    "./data", 
    file_metadata=filename_fn, 
    recursive=True
).load_data()

documents has this :

Attachments

2 comments

WWhiteFang_Jr

No they are not removed by default.
You can remove if you dont want the embed_model and llm to see the metadata.

Plain Text

# once you have the docs, you can iterate like this to remove the metadata keys
for doc in documents:
    doc.excluded_embed_metadata_keys = []
    doc.excluded_llm_metadata_keys = []

gghxsted.

ok thanks ;))

Add a reply