The Importance of Accurate Ground Truth Data for Effect...

At a glance

The community members are discussing the requirements for a 4-6 page quarterly report. One community member has a template for a report structure, while another has an example report that incorporates facts from monthly reports. The community members emphasize the need for a process to generate the report and review the facts, potentially using language models like GPT-4. They also discuss the use of Retrieval-Augmented Generation (RAG) techniques to improve the report generation, and a community member shares a repository with various RAG implementations that could be helpful.

Useful resources

NNxTdoor

essentially it needs to be a 4-6 page report, we have one quarter's ground truth example

11 comments

LLogan M

Do you have a template in mind for that the report should look like?

For example, I'm working on a report gen api. I provide a template and some files. My template looks like

Plain Text

# Latest Papers on RAG

## TLDR
A quick summary of the latest papers.

## Details
Details for each paper, ideally with a section per paper.

NNxTdoor

I DO HAVE yes

NNxTdoor

well there's an example report

NNxTdoor

it doesn't have a structure but it incorporates all facts from monthly reports

LLogan M

I think you'll need a process to both generate report, and then also review the facts once its generted with another LLM process

NNxTdoor

More than report structure or template, we care about bringing together all facts and important insights in quarterly reports, for whatever was mentioned in monthly reports

NNxTdoor

yea some sorta function calling loop

NNxTdoor

what are you doing for ur usecase

NNxTdoor

current methods miss facts, often hallucinate and fail to include enough details ... we expect a detailed report , is it possible to generate with gpt-4/4o

LLogan M

It definitely is, but you need proper reflection (and probably also decent RAG or file selection for filling in certain sections of the report)

My approach tries to take a template + source files, and make a plan of how to fill in that template

PParas Madan

So I have been playing around with RAG for a while now and the best way to try out what works best on your data is to try different types of RAG solutions. I found this amazing repo on Hacker News which has 10+ RAG Implementations along with google collab notebooks. They implemented Naive RAG, Hyde RAG, Hybrid RAG, Corrective RAG, Self RAG and 5+ other techniques. Check out: https://github.com/athina-ai/rag-cookbooks

Thought it would be helpful for everyone.

Add a reply

Find answers from the community

The Importance of Accurate Ground Truth Data for Effective Reporting