The community member is looking to create a custom sentence splitter for an ingestion pipeline, where they want to split text primarily based on a regex separator rather than chunk size. They ask if they can use the existing sentence splitter or if they need an alternate approach. A comment suggests that it may not be possible with the existing sentence splitter, and the community member would need to subclass and create their own custom splitter. The comment also mentions checking the existing sentence splitter implementation as an example to follow.
Question about customs transformations with ingestion pipelines. I am looking to create a custom sentence splitter type thing where i am splitting primary on a regex separator vs primarily on chunk size. Can i use sentence splitter or do I need an alternate? any examples of something similar?
