File loaders

The file loaders just return the data from files, so that you can index it with llama index.

Then you can ask questions to your index 👍

thanks @Logan M
should the file loader do any pre-processing than?
org-mode, which I want to parse, is very hierarchical. Like json.
I don't want children to be orphaned for search. If I search for "Peters Birthday" it has to find the birthday node under peter.

I finally managed to load my file generically 🙂
but I feel the lack of context right away...

Yea that's understandable! It's a tricky thing.

You could build a smarter parser that tries to maintain the hierarchical information.

Like parse each child node into a string that contains the parents

people -> Peter -> birthday -> April 7, 2023

Just a rough idea though, but something like that 🤔

yea thats what i thought (and hoped the json parser does and i just need to fork it...)

I'd also love to add weights to the chunks: older = less important

I think the latest json loader actually does this, was just reading more about it yesterday 😅

Time filters/weights hopefully coming soon🙏