thanks. Do you happen to know how do I create graph for one markdown document, with header as parents, and list nodes capture the paragraph under the header?
Sounds like you just need to create a custom markdown parser that creates the Document
objects the way you want to π€·ββοΈ
Is there a tutorial or something that I can learn from?
@Logan M thank, I will take a look. And by the way, maybe it is just me, but I think retrieval is such a core part of what llamaindex do, it feels like it deserve a separate section from "query module".
and leave the query/chat engine as high level interface,
The above video focuses on data loading though. Not sure which part you meant to refer to haha
where is the example where I can construct the tree structured nodes (graph) based syntax of document? @Logan M
I think what I am trying to say is: retrieval is low level, and query engine is high level.
I understand the high level api will help many, but the core value of llamaindex is low level.
So at least for me, by separate these two in the documentation into two sections, it is easy for me to focus.
Also, it will be better if query engine and chat engine are built on top of retriever instead of index.
I know I am being too picky.
I know, all the example in the doc says query engine can be get from index...
I understand, but that one line short hand is actually not helping in understanding how things work.
I have to think to get it.
Any how, you guys might have a different priority.
Can you point me to some example where I can build some structure on the list of node? Also the, if there are only next/prev, how to we build the tree structure?
I think that low level api example is exactly what you are looking for π
In actuality, every component in llama-index is customizable
We follow a thing called progressive disclosure -- simple API at the start, but you are still able to drill down and pick things into base components (retriever, response synthesizer, node postprocessor...)
There is prev/next and parent/child. These relationships are not really exploited at all though unlesss you've built a custom retriever or custom node-postprocessor
That code sample above for the markdown file makes every code block a child of the parent text block for example. There's not automatic way to do this, you have to build the structure yourself as you parse the file though
understand, my issue really is: by get query from index, you are expose the link between query and index, but hide retrieval that is in between.
just my two cents, it is better to create retriever (not index) from the disk, and get query/chat engine from retriever, there is no gap this way.
Again, I am trying to be picky.
but at the same time, I feel the llamaindex's core value is index/retrieval.
indexing produce index which retriever will user, which query engine will use. π
@Logan M thanks for the help.