I still have some questions about LlamaIndex, as I am unsure about the appropriate use cases for it. Please allow me to list my concerns:
- I understand that LlamaIndex involves splitting text into chunks, obtaining embedding information, and then storing that along with metadata in a JSON file. As this process accumulates a significant amount of data, would it be difficult to use this method for services that require extensive data storage?
- If scalability is indeed a challenge due to reason 1, in what situations would LlamaIndex be most effectively utilized?
- When loading text data from a database, I presume that no vector-based index is created. How, then, does LlamaIndex efficiently search for relevant information in response to a query?
- If searching external sources proves to be slow due to reason 3, when would it be advantageous to use LlamaIndex?
My stance is that I see potential in LlamaIndex, but some questions have arisen as I've been researching. I appreciate your assistance in addressing these concerns.