Are there any ways currently to alter the underlying prompts and flows within some of the query engines or retrievers? Like altering the prompt for how the sub questions created in the sub question retriever, how the answers are synthesized? Thanks!

Does anyone have any advice or experience or resources for ‘ thresholding’ the vectors you retrieve from RAG? What I mean is, I don’t want to just retrieve the top 10 most relevant and stuff/map-reduce whatever on all of them, if only 3 are super relevant. Likewise, maybe 20 vectors are all super crucial, in which case I want all of them

Hi @Logan M @jerryjliu0 wanted to ask you if there's any guides or useful references for RAG on semi-structured data. Or rather, a combination of structured + unstructured data. I have a use case in which we're taking a lot of unstructured data, and then creating a semi-structured representation of it: Some concrete quantitative/categorical fields that can be used for filtering in traditional database queries, and then some more descriptive, textual fields. The way we're envisioning the end use is: Based on user queries, doing some initial database filtering, then being able to do RAG on the descriptive, textual fields that meet those filtering criteria. Do you think there any guides or resources that could help with this, internal or external to Llamaindex? Thanks so much!

Would GPT Index be appropriate for this? And if so, how would I provide the previous conversational responses as continuing context to the LLM, as well as enable it to make these functional calls internally within the GPT Index architecture?