Find answers from the community

Updated 2 years ago

is it possible to run multiple distinct

is it possible to run multiple distinct queries in parallel?
I've noticed that aquery seems to block execution, even with asyncio code

Plain Text

    async def query(i):
        return await query_engine.aquery(...)

    tasks = [query(i) for i in range(10)]
    sections = await asyncio.gather(*tasks)

7 comments

mmkern

hey @jerryjliu0, please excuse the ping, I'm trying to deploy an app to production and this is kind of the major last missing piece. are queries designed to be isolated right now? This example was using a tree index, and queries are always ran sequentially even though the code should allow for concurrency

jjerryjliu0

Hey @mkern , apologies for the delay. 1) why are you using a tree index (try using vector store index), and 2) assuming you're using a vector index try doing query_engine = index.as_query_engine(response_mode="tree_summarize")

jjerryjliu0

the tree summarize calls have full async support

jjerryjliu0

whereas the default ones do not atm (which we can fix!)

mmkern

hey! thanks for getting back here! I am experimenting with a tree index, as I wanted to categorize nodes into certain groups automatically (or even have user defined groups).

mmkern

I do understand that a vector index would be likely much more time and cost efficient, but I was still curious about the qualitative differences

mmkern

I'll look into the new document summary index too, that seems like the best of both worlds?

Add a reply