Find answers from the community

Updated 2 years ago

Text splitting

awesome! I was wondering if there are Any best practices for choosing the appropriate node length? IMHO a larger window may adds too much noise yet a small window might lose context info/
L
R
5 comments
My best advice is to try and group things into logical groups
Like, if you have clear sections that you can parse out, those make good document splits
Otherwise, overlapping chunks usually works ok-ish
I see. I guess I can use Langchain's text splitter to build nodes and use Llama index, since they offer a few different ones.
Yea for sure!
Add a reply
Sign up and join the conversation on Discord