Find answers from the community

Updated 2 months ago

<@&1075540741033566271> How do Load only necessary data based on the query?

How do Load only necessary data based on the query?
k
3 comments
Please stand by @Joey as I search the knowledge base. I normally take about 20 seconds or so. :wave:
----------------------
To load only the necessary data based on a query in LlamaIndex, you would typically use a query engine. The query engine is responsible for selecting the relevant data based on the query.

For example, in the Joint QA Summary Query Engine, the router query engine selects the relevant query engine based on the context of the documents needed to answer the question. This is demonstrated in the following code snippets:

Plain Text
INFO:llama_index.query_engine.router_query_engine:Selecting query engine 0 because: This choice is relevant because it involves retrieving specific context from documents, which is needed to answer the question about the author's activities growing up..

Plain Text
INFO:llama_index.query_engine.router_query_engine:Selecting query engine 0 because: This choice is relevant because it involves retrieving specific context from documents, which is needed to answer the question about the author's activities in art school..

In these examples, the query engine is selecting the necessary data (documents) based on the query about the author's activities.

For loading data, you can use the SimpleDirectoryReader to load data from a local directory as shown in the following code:

Plain Text
from llama_index import SimpleDirectoryReader

# try loading great gatsby

documents = SimpleDirectoryReader(
    input_files=["./gatsby_full.txt"]
).load_data()

In this example, the SimpleDirectoryReader is used to load the data from the file gatsby_full.txt.
For more information, you can refer to the LlamaIndex documentation () and the examples provided ().
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Add a reply
Sign up and join the conversation on Discord