Hi all, I am trying to create a GPT labeled dataset by reading a file and getting its summary as my output. I have about 1K files to summarize. Is there a way to pass all at once or do I have to continue looping each time for each file. The prompt for all files is the same. The below is what I have and it seems to combine all the documents into 1
I've been meaning to make a PR to make a sort of response_mode="accumulate" option, so that a single prompt as applied across all documents. Haven't had time yet lol