Can TransformQueryEngine take multiple

SSimon 📐🛠

Can TransformQueryEngine take multiple query engines (array of engines)? I asked the LLama chatbot and it provided a response that doesn't work:

Plain Text

from llama_index.indices.query.query_transform.base import HyDEQueryTransform
from llama_index.query_engine.transform_query_engine import TransformQueryEngine

# Initialize your index and query engines
query_engine_1 = YourQueryEngine1()
query_engine_2 = YourQueryEngine2()

# Define your query transformation
hyde = HyDEQueryTransform(include_original=True)

# Create an array of query engines
query_engines = [query_engine_1, query_engine_2]

# Create a TransformQueryEngine with the array of query engines and query transformations
transform_query_engine = TransformQueryEngine(query_engines, query_transforms=[transform_1, transform_2])

# Query the index using the TransformQueryEngine
response = transform_query_engine.query("Your query")

# Process the response
# ...

10 comments

LLogan M

Yea it only takes a single query engine

Attachment

LLogan M

What's the use case for giving it multiple query engines? 🤔

SSimon 📐🛠

Following our last discussion I am finding HyDE really accurate and so I want to use it over many Documents (a csv file, a pdf file, etc.).
So I guess I need to be adding these multiple Documents under the one engine first?

SSimon 📐🛠

These file formats are originally from the one original pdf file, which I would prefer to keep that relationship, so maybe I should be focusing on how I am reading the pdf. For instance, read the pdf, if there is a table read that but if it needs to unmerge cells then do this before adding it to the final Document. Then create a VectorIndex from this Document.

LLogan M

Correct! Or, you can wrap each of your sub-indexes with a transform engine as well

LLogan M

Yea, I think that makes sense 🤔

SSimon 📐🛠

How do you create a custom Reader? Do you have this in your Docs?

LLogan M

Not quite, but to be honest it's not anything special

If you want to eventually add it to llama-hub, you could look at some simple examples from there (I can link in a second, github is not loading for me haha)

But at the end of the day, you can just implement a function that takes in some input (URLs, files, etc.) and returns a list of Document objects

LLogan M

Here's a simple one from llama-hub: https://github.com/emptycrown/llama-hub/blob/main/llama_hub/file/flat_pdf/base.py

SSimon 📐🛠

Thanks!

Add a reply

Find answers from the community

Can TransformQueryEngine take multiple