Huge work! Do you think that the this new lower level API (node) also affects a big text db or only json, images etc? Can you better explain the implications?
yeah so the new lower-level API allows you to define "chunks" individually instead of feeding in an entire Document object. This chunk could be a text chunk, a single image, or more. We're still figuring out the right abstractions on our end, so stay tuned for how this affects multimodal data (e.g. images + text) and semi-structured data (e.g. json)
Im working with texts chunks. Do you suggest to work with documents or nodes? If nodes are able to "get relations" with other nodes (eg the previous one and the following one) I would say they are better! But I need time to get an understanding of all new features
yeah Nodes are the new lower-level API so if you wanted to, would recommend that you try manually parsing docs into nodes (through NodeParser) or constructing Nodes manually!
Thx @jerryjliu0. Still, I can't see the advantages in using the nodes. Are the next versions of llama_index based on them? What can I gain in terms of performance (speed, reliability...)? The guide is fantastic but not too explanatory tbh! I mean, I appreciate what you do and I hope you take this message as a user feedback! I see other users keeping using the older version, maybe we all need a clearer explanation in order to switch to the new version. Also because I created my code using the old version and switching is time expansive, but at the same time I wouldn't lose any updated on this amazing work.
@AndreaSel93 it's a good point. we don't have a ton of features that take advantage of these refactors so far (besides the fact that you can now manually define nodes / parse them yourselves instead of trusting us to handle it)...but we will soon!
