Automated retrieval with llama-index blocked. Is there a work-around?
from llama_index.core import SummaryIndex
from llama_index.readers.web import SimpleWebPageReader
from IPython.display import Markdown, display
import os
documents = SimpleWebPageReader(html_to_text=True).load_data(["https://www.xyz.com"])
documents
[MyHomepage] Main\nContent Main Navigation\n\n## Page not available\n\n
Your access to website has been blocked because you are using an\nautomated process to retrieve content\n\nReason: Automated retrieval by user agent "python-requests/2.31.0".\n\nURL:
