@here I see **Sub Question Query Engine

my guy, please don't @ - here

You can use a question generator to generate questions and gather sub responses

Thank you @Logan M for quick response. One additional question:
if we have 100 pdfs then shall we need to create seperate tool for each pdfs?

You could, but that probably won't scale well. You probably want to retriever some top-k of documents that match the query, and then generate questions from there (like, top-10 for example).

You could do this by embedding descriptions for example

ok but seems using only 1 query tool will be good if we are considering scaling also.

below is snippets:

input_dir = './data/pdf_directory/'
pdf_nw = SimpleDirectoryReader(input_dir=input_dir).load_data()

build index and query engine

vector_query_engine = VectorStoreIndex.from_documents(
pdf_nw,
use_async=True,
).as_query_engine()

setup base query engine as tool

query_engine_tools = [
QueryEngineTool(
query_engine=vector_query_engine,
metadata=ToolMetadata(
name="pdf_nw",
description="multiple pdfs i HAVE WOrked on",
),
),
]

question_gen = LLMQuestionGenerator.from_defaults(llm=llm)
sub_questions = question_gen.generate([tool.metadata for tool in query_engine_tools], QueryBundle(query))

for question in sub_questions:
print(question.sub_question)


Now seems in this case it makes sense to creatre only 1 query_engine_tools if we have 100s of pdfs?