👋 Let s say I want to summarize a

At a glance

👋 Let's say I want to summarize a single technical paper and theorize that the summary would be improved if the LLM was augmented with the remainder of the domain specific corpus (e.g. all the other papers published in the journal). How would I go about testing that theory? The Summarization over Documents example (https://gpt-index.readthedocs.io/en/latest/guides/use_cases.html#use-case-summarization-over-documents) using GPTTreeIndex clearly documents how to summarize the entire corpus, but I'm not understanding yet how to summarize a single index/Document while also augmenting with a larger index. Perhaps, I have a fundamental misunderstanding. I'm looking forward to learning from the group and hope to contribute in the future.

2 comments

jjerryjliu0

just for me to understand, how exactly would you "augment" with the domain corpus? would you want to look up specific chunks of that corpus to augment the summary or also look over the entire thing?

sstevensurgnier

Thanks for taking the time to answer my question and the great library! I would also want to look over the entire thing. Part of the motivation of the theory is that the LLM hasn't been trained on the entire corpus and the summary of a single paper may benefit if it had been. Perhaps, that's a better way to describe what I'm trying to accomplish.

Add a reply

Find answers from the community

👋 Let s say I want to summarize a