Find answers from the community

Updated 2 months ago

Text splitter

do we have any other text preprocessor apart from textsplitter and recursivetextsplitter?
for one document textsplitter works good and sometime on other recuresivetextsplitter works good, sometimes neither of them.
cc: , ,
L
1 comment
You can use any text splitter that langchain has.

Tbh I wouldn't worry too much about how the text is being split. Since there's a large amount of text overlap when injesting documents (default 200 tokens), the splits end up not mattering as much
Add a reply
Sign up and join the conversation on Discord