is it possible to run multiple distinct

At a glance

The community member is trying to run multiple distinct queries in parallel, but has noticed that aquery seems to block execution, even with asyncio code. Another community member suggests using a vector store index instead of a tree index, and trying query_engine = index.as_query_engine(response_mode="tree_summarize"), as the tree summarize calls have full async support, unlike the default ones. The original community member is experimenting with a tree index to categorize nodes into groups, and is interested in the qualitative differences compared to a vector index, but acknowledges that a vector index would likely be more time and cost efficient. They also mention looking into the new document summary index as a potential best-of-both-worlds solution.

mmkern

is it possible to run multiple distinct queries in parallel?
I've noticed that aquery seems to block execution, even with asyncio code

Plain Text

    async def query(i):
        return await query_engine.aquery(...)

    tasks = [query(i) for i in range(10)]
    sections = await asyncio.gather(*tasks)

7 comments

mmkern

hey @jerryjliu0, please excuse the ping, I'm trying to deploy an app to production and this is kind of the major last missing piece. are queries designed to be isolated right now? This example was using a tree index, and queries are always ran sequentially even though the code should allow for concurrency

jjerryjliu0

Hey @mkern , apologies for the delay. 1) why are you using a tree index (try using vector store index), and 2) assuming you're using a vector index try doing query_engine = index.as_query_engine(response_mode="tree_summarize")

jjerryjliu0

the tree summarize calls have full async support

jjerryjliu0

whereas the default ones do not atm (which we can fix!)

mmkern

hey! thanks for getting back here! I am experimenting with a tree index, as I wanted to categorize nodes into certain groups automatically (or even have user defined groups).

mmkern

I do understand that a vector index would be likely much more time and cost efficient, but I was still curious about the qualitative differences

mmkern

I'll look into the new document summary index too, that seems like the best of both worlds?

Add a reply

Find answers from the community

is it possible to run multiple distinct