Hi there. I have a use case for my engine, which is essentially to store all documents as a list index, then fire use the entirety of that index, i.e. all nodes without any filtering - how best to achieve this? - Also, is there a way I can use the LLM to filter if I wanted to? use case is basically to put a chat inteface infront of a monitoring solution, so I envisage a two step process - 1) use the LLM to extract useful stuff from loads of logs 2) send the relevant logs to the LLM along with the users view of the symptoms. I also think I may have just described the base functionality of llama-index 😄
Although if you want to extract structured data, you might be interested in a pydantic program. We recently used it to analyze our guthub issues and pull out common issue types and then summarize
