Find answers from the community

Updated 10 months ago

Metadata

Hi Guys
I use OpenAI and service_context with nodes
and when the data is retrieved from indexing they send to llm with metadata like
page_label and file_path, so i want to send just the context without this metadata , how i can do it?

4 comments

LLogan M

When you load your documents, set the excluded metadata keys

document.excluded_llm_metadata_keys = ["page_label", ...]

kkemoalkoul

Can u clarify where must do this step?
This is my code

Plain Text

documents = SimpleDirectoryReader("/content/data").load_data()

node_parser = SimpleNodeParser.from_defaults(chunk_size=512, chunk_overlap=20)
nodes = node_parser.get_nodes_from_documents(documents)

LLogan M

Could be either on the documents from the reader, or the nodes from the parser, either would work

kkemoalkoul

thank u

Add a reply