Find answers from the community

Updated 8 months ago

Parse

This is hopefully a simple question. I have parsed a markdown file into nodes using the markdown parser but some of these nodes are too big still. How do I further chunk these while preserving most of the metadata? Is this something I have to write a function for or is there an easy way to do this already in the library?
L
b
3 comments
You can just pass those nodes to another node parser / text splitter

Plain Text
nodes = ....

splitter = SentenceSplitter(chunk_size=1024, chunk_overlap=20)
nodes = splitter(nodes)
Ah that was simple lol
Add a reply
Sign up and join the conversation on Discord