NeroBot - The LLM powered scraper for yo...

kkagnar

Hi everyone! We are going to be live in closed beta pretty soon for my new project NeroBot, a next generation web crawler to support Enterprise RAG apps. We have been very inspired, from early on, by the LlamaIndex framework and research, thank you team. (hopefully this is the write channel)

Check out our website and get on the waitlist here: https://www.nero.ai/

Also i wrote a brief post outlining the story for why we are building: https://blog.nero.ai/why-were-building-nerobot-the-next-generation-crawler-784e5150f3bb

6 comments

TTeemu

Wow this is super cool! Wouldn't be surprised if we saw more of these types of applications released soon 👀

kkagnar

Thank you! appreciate the feedback

TTeemu

How do you manage to handle pop ups and stuff? You using LLMs for that also?

kkagnar

Pop ups don’t block html, so we use llms to find the div of the page content. If there is some type of content obfuscation, we wouldn’t be able to scrape.

TTeemu

Ah I just assumed you we're also using some automated browser software in the background

TTeemu

But yeah that makes sense

Add a reply

Find answers from the community

NeroBot - The LLM powered scraper for yo...