Answer metadata

That metadata is a column in your csv?

Try checking the source nodes from the response object

response.source_nodes gets a list of every node used to create the response

@Logan M how can I control columns? For example I have 3 columns: content, link and title and want display only 2 of them.

Yeah, and also this output doesn't provide info about my metadata at all:(

How did you load the csv?

Here is a code: @Logan M

SimpleCSVReader = download_loader("SimpleCSVReader")

loader = SimpleCSVReader()
documents = loader.load_data(file=Path('output_file.csv'))

max_input_size = 4096
max_chunk_overlap = 20
num_output = 512
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="gpt-4", max_tokens=num_output))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper, chunk_size_limit=2500)
index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)

That csv loader will create a document object for each row in the csv, just by concatenating everything

You might be also interested in the "PagedCSVReader", which will do something similar but include the column names

If there's extra info you want to store, you can use the extra_info field of each document

document.extra_info = {"Some key": "Some val", ...}

Oh, got it. I will try, thank you:)

@Logan M What if I load Data from langchain (which consider about metadata) and after will use Document.from_langchain_format(langchain_document) to convert it to gpt index. Will it work?

Yea, that will set the extra_info field for you. Then that info should show up in the source nodes

@Logan M but some reason I've got this:

ValueError: nodes must be a list of Node objects.

What line of code throws that error? Did you still use the from_documents method to create the index?

I fixed this:) @Logan M

Thank you so much:)

@Logan M I've tried to use PagedCSVReader, but can't undestand how to add extra_info to each documents based on my columns:(

Hmm. That's annoying.

Maybe it's best to just manually load the sheet yourself and create document objects, so it's formatted the way you want?

I solve it:)

I literally re-write whole langchain class CSV loader))) @Logan M

Hahaha nice!