Find answers from the community

Updated 9 months ago

Hello, I have a simple question! is the

Hello, I have a simple question! is the meta data excluded by default in the embeddings generation and LLM?
this is my code :
Plain Text
from llama_index.core import SimpleDirectoryReader

filename_fn = lambda filename: {"file_name": filename}

# automatically sets the metadata of each document according to filename_fn
documents = SimpleDirectoryReader(
    "./data", 
    file_metadata=filename_fn, 
    recursive=True
).load_data()

documents has this :
Attachments
image.png
image.png
W
g
2 comments
No they are not removed by default.
You can remove if you dont want the embed_model and llm to see the metadata.

Plain Text
# once you have the docs, you can iterate like this to remove the metadata keys
for doc in documents:
    doc.excluded_embed_metadata_keys = []
    doc.excluded_llm_metadata_keys = []
Add a reply
Sign up and join the conversation on Discord