Find answers from the community

Updated 5 months ago

what would be the best node parser to

what would be the best node parser to use in the case where I have markdown files with tables inside ? MarkdownElementNodeParser or MarkdownNodeParser ?
Can anyone please explain the difference is using these two parsers?

Difference in terms of:
Indexing strategies
Influence on the retrieval part of the rag pipeline in terms of performance ?

an example of document i have would be this :
W
g
2 comments
MarkdownElementNodeParser along with splitting the text it uses llm to create summary of the extracted content.

This gives an extra edge to this parser.

Whereas MarkdownNodeParser only converts the markdown content into nodes.
I read it generates two types of nodes:
Text Nodes and Index Nodes corresponding to embedded objects (e.g. tables)
how does it handle tables in the markdown document? does it create a node with the "text" of the table and a summary of what it's about using the llm ?
Add a reply
Sign up and join the conversation on Discord