Hi, I'm using 'CodeHierarchyNodeParser'

Hi, I'm using 'CodeHierarchyNodeParser' and its been great. I was testing my code with several python files and one give me this error 'string index out of range'. Following the traceback and looking the file I'm trying to parse I think that is because one variable definition has a reaally long string. My code is this:

Plain Text

documents = SimpleDirectoryReader(
            input_files=[path],
            file_metadata=lambda x: {"filepath": x},
        ).load_data()

        code = CodeHierarchyNodeParser(
            language=self.language,
            chunk_min_characters=0,
            code_splitter=CodeSplitter(language=self.language, max_chars=10000, chunk_lines=10),
        )
        no_extension_path = self.file_path.replace(".py", "")

        split_nodes = code.get_nodes_from_documents(documents)

How can I fix this?

Find answers from the community

Hi, I'm using 'CodeHierarchyNodeParser'