Find answers from the community

Updated 3 months ago

Hey everyone I am trying to ingest data

Hey everyone! I am trying to ingest data that i have in a csv. I looked at the pandas csv loader and it seems to me it puts all of the columns into the “text” argument of a Document. I am trying to figure out if it is best practice to do that or if non-text columns should be put in “extra_info”. I am wondering tho if that extra_info gets embedded as well when writing the data to a vector store
d
1 comment
you can customize how metadata is handled in when it's passed to LLM for completion/embedding. Checkout this guide: https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_documents.html
Add a reply
Sign up and join the conversation on Discord