Find answers from the community

Updated 6 months ago

Hi all,

Hi all,
I believe theres a bug when you use SentenceWindowNodeParser with a custom SentenceSplitter.

Plain Text
sentence_splitter = SentenceSplitter.from_defaults(
    secondary_chunking_regex=r"[^,.;。?!\n]+([,.;。?!\n]|\.{2,})?"
)

# Initialize the SentenceWindowNodeParser with the custom SentenceSplitter
node_parser = SentenceWindowNodeParser.from_defaults(
    window_size=3,
    window_metadata_key="window",
    original_text_metadata_key="original_text",
    sentence_splitter=sentence_splitter
)
nodes = node_parser.get_nodes_from_documents(documents)


I get an error
AttributeError: 'str' object has no attribute 'id_'

In the node_parser step, when I do not give a sentence splitter everything works as intended. Can someone help out?
L
D
2 comments
The naming is maybe confusing, but sentence_splitter is supposed to be a function, not a SentenceSplitter object

sentence_splitter=sentence_splitter.split_text might work
Add a reply
Sign up and join the conversation on Discord