Find answers from the community

0
0ptim
Offline, last seen 3 months ago
Joined September 25, 2024
I use SimpleWebPageReader to scrape this page https://www.defichainwiki.com/docs/auto/App_Sync_Boost.

Then I index it with GPTListIndex(documents) which throws A single term is larger than the allowed chunk size.Term size: 7103Chunk size: 3714.

I think it must be because of the image which the scraper loads as base64 string (message.txt).

How to work around this issue?
6 comments
v
y
0