Find answers from the community

Updated 9 months ago

If i don't use aquery or use_async=True

If i don't use aquery or use_async=True, would asyncio not consider this task async?

Plain Text

        prompt = self._prompt(municipality_dto.name)
        prompt = re.sub(
            r"\s{3,}",
            " ",
            prompt,
        )

        response_synthesizer = CompactAndAccumulate(
            text_qa_template=CUSTOM_TEXT_QA_PROMPT,
            output_cls=ContactList,
        )

        query_engine_contact = index.as_query_engine(
            response_synthesizer=response_synthesizer
        )

        contact_list: PydanticResponse = query_engine_contact.query(prompt)  # type: ignore

13 comments

bbmax

vs this:

Plain Text

        response_synthesizer = CompactAndAccumulate(
            text_qa_template=CUSTOM_TEXT_QA_PROMPT,
            output_cls=ContactList,
            use_async=True,
        )

        query_engine_contact = index.as_query_engine(
            response_synthesizer=response_synthesizer
        )

        contact_list: PydanticResponse = await query_engine_contact.aquery(prompt)  # type: ignore

excuse me, i'm not on .1 yet

LLogan M

Yea use_async here is just controlling some underlying behaviour

In each case, the top-level func is sync

LLogan M

Since accumulate uses the same prompt for each chunk, it can distribute them across many async calls under the hood using asyncio.run(asyncio.gather(...))

bbmax

so don't need aquery

LLogan M

correct. Not unless you need async at the top level function

yeah for some reason

use_async=true

hurts

https://community.openai.com/t/error-in-using-async-client-of-openai-when-i-use-custom-http-client-with-httpx/582525