How do i make async call using RAGStringQueryEngine?

3 comments

Please stand by @Joey as I search the knowledge base. I normally take about 20 seconds or so. :wave:

----------------------
The extracts provided do not contain specific information on how to make asynchronous calls using RAGStringQueryEngine. However, there is an example of making asynchronous calls with a list of queries using a generic query_engine in LlamaIndex. Here's the example:

Plain Text

import asyncio
import time

# a list of different queries
query_list = [query_str] * 3

start_time = time.perf_counter()
query_engine = index.as_query_engine(
    response_mode="tree_summarize",
)

# run each query in parallel
async def async_query(query_engine, questions):
    tasks = [query_engine.aquery(q) for q in questions]
    r = await asyncio.gather(*tasks)
    return r

_ = asyncio.run(async_query(query_engine, query_list))
elapsed_time = time.perf_counter() - start_time

print(f"{elapsed_time:0.3f}s")

(Source: Async Query Demo ())

In this example, asyncio.gather is used to run multiple queries in parallel. You can replace query_engine with an instance of RAGStringQueryEngine and use the aquery method for asynchronous calls. Please note that this is a general approach and might need adjustments based on the specific requirements of RAGStringQueryEngine.

For more detailed information, you may want to refer to the LlamaIndex documentation () or the LlamaIndex GitHub repository ().
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

rrahul

@Joey The aquery method is defined in BaseQueryEngine and should be inherited by RAGStringQueryEngine. However, RAGStringQueryEngine needs to implement the _aquery method, which is called by aquery. Here's how you might do it:

Plain Text

class RAGStringQueryEngine(BaseQueryEngine):
    ...
    async def _aquery(self, query_bundle: QueryBundle) -> RESPONSE_TYPE:
        query_str = query_bundle.query_str
        nodes = await self.retriever.retrieve(query_str)
        context_str = "\n\n".join([n.node.get_content() for n in nodes])
        response = await self.llm.complete(
            qa_prompt.format(context_str=context_str, query_str=query_str)
        )
        return str(response)

Then, you can use aquery like this:

Plain Text

engine = RAGStringQueryEngine(...)
response = await engine.aquery("What is the capital of France?")
print(response)

Replace ... with the necessary arguments to create an instance of RAGStringQueryEngine.

Add a reply

Find answers from the community

How do i make async call using RAGStringQueryEngine?