Hi Team, In the above screenshot, some

If your documents contain metadata, then by default text + metadata will got for response generation.

You can customize this though by adding the metadata keys you dont want LLM to see for generating the response.

https://docs.llamaindex.ai/en/stable/module_guides/loading/documents_and_nodes/usage_documents.html#customizing-llm-metadata-text

We are creating top_nodes with only texts using the NodeWithScore class

After semantic search, we are sending the useless metadata to OpenAI. We want to remove the unnecessary metadata. Here,
Can you confirm if these were sent by Llamaindex?

node metadata that does not need to be sent to openai :-
admin_id
provider
timestamp
chunk_size
chunk_overlap
num_indexes
prompt_id
object_id

No i dont think these values are being passed from LlamaIndex side.

Actually,In typesense and Pinecone search include this metadata. And a lot of useless metadata is also sent to OpenAI with the source node contexts as I mentioned in Screenshot[portkey SS]

is it possible to remove this?[before sending to OPENAI]

Just make sure your nodes don't have metadata that you don't want to send, and it should be fine? Although tbh I'm pretty confused by your setup lol

I'm guessing this metadata is being introduced by some tool after retrieving the top k? So just need to pre-process those nodes