Find answers from the community

Updated 3 months ago

NeroBot - The LLM powered scraper for yo...

Hi everyone! We are going to be live in closed beta pretty soon for my new project NeroBot, a next generation web crawler to support Enterprise RAG apps. We have been very inspired, from early on, by the LlamaIndex framework and research, thank you team. (hopefully this is the write channel)

Check out our website and get on the waitlist here: https://www.nero.ai/

Also i wrote a brief post outlining the story for why we are building: https://blog.nero.ai/why-were-building-nerobot-the-next-generation-crawler-784e5150f3bb
T
k
6 comments
Wow this is super cool! Wouldn't be surprised if we saw more of these types of applications released soon 👀
Thank you! appreciate the feedback
How do you manage to handle pop ups and stuff? You using LLMs for that also?
Pop ups don’t block html, so we use llms to find the div of the page content. If there is some type of content obfuscation, we wouldn’t be able to scrape.
Ah I just assumed you we're also using some automated browser software in the background
But yeah that makes sense
Add a reply
Sign up and join the conversation on Discord