Hey guys! I have doubts about modeling a

llucasmsoares96

Hey guys! I have doubts about modeling a system with RAG. My data comes from a website that I collected using WholeSiteReader, a 100,000-line JSON that comes from an API and possibly an Oracle database.

Using only the vector database I didn't have a good result on my agent, so I decided to try the knowledge graph. But it is taking too long to build this graph (hours) and it is getting very expensive. Is this right? Is there any more efficient way to do this?

I thought about using ReActAgente.from_tools, with a tool for the JSON Query Engine and another for the knowledge graph. Is this a better approach?

3 comments

LLogan M

Probably knowledge graph is not the approach here

Instead, its more about processing your data into documents/chunks that make sense

On top of that, using retrieval techniques like hybrid and reranking, along with sub-question or agents, would help

LLogan M

The bigger the dataset, the more techniques you need to throw at it, and the more time you need to spend properly ingesting data

llucasmsoares96

@Logan M In fact, I think I need the knowledge graph. I'm doing webscraping in an web store, with a lot of related information that is scattered on several pages.

Add a reply

Find answers from the community

Hey guys! I have doubts about modeling a