how should I async query>

5 comments

Please stand by @Joey as I search the knowledge base. I normally take about 20 seconds or so. :wave:

----------------------
To perform an asynchronous query using LlamaIndex, you can use the aquery method of the query_engine object. Here's a step-by-step guide:

Import necessary modules and apply nest_asyncio if you're using Jupyter notebook:

Plain Text

import nest_asyncio
import time
from llama_index import SummaryIndex, SimpleDirectoryReader

nest_asyncio.apply()

Load your documents:

Plain Text

documents = SimpleDirectoryReader("../paul_graham_essay/data").load_data()

Create an index from your documents:

Plain Text

index = SummaryIndex.from_documents(documents)

Create a query engine with use_async=True:

Plain Text

query_engine = index.as_query_engine(use_async=True)

Run your query using aquery method:

Plain Text

response = await query_engine.aquery("Your query string here")

Remember to replace "Your query string here" with your actual query.

For more details, you can refer to the Async Query Demo () on the LlamaIndex GitHub page.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

JJoey

class RAGStringQueryEngine(CustomQueryEngine):
"""RAG String Query Engine."""

retriever: BaseRetriever
response_synthesizer: BaseSynthesizer
llm: OpenAI
qa_prompt: PromptTemplate

def custom_query(self, query_str: str):
nodes = self.retriever.retrieve(query_str)

context_str = "\n\n".join([n.node.get_content() for n in nodes])
response = self.llm.complete(
qa_prompt.format(context_str=context_str, query_str=query_str)
)

return str(response)

@kapa.ai how do i modify this to make async calls

kkapa.ai

Let me check the knowledge base for you. One moment! :books:

kkapa.ai

----------------------
To modify the RAGStringQueryEngine class to make asynchronous calls, you would need to define an asynchronous method acustom_query in the class. Here's how you can do it:

Plain Text

class RAGStringQueryEngine(CustomQueryEngine):
    """RAG String Query Engine."""

    retriever: BaseRetriever
    response_synthesizer: BaseSynthesizer
    llm: OpenAI
    qa_prompt: PromptTemplate

    async def acustom_query(self, query_str: str):
        nodes = await self.retriever.retrieve(query_str)

        context_str = "\n\n".join([n.node.get_content() for n in nodes])
        response = await self.llm.complete(
            qa_prompt.format(context_str=context_str, query_str=query_str)
        )

        return str(response)

Please note that this modification assumes that the retrieve and complete methods of retriever and llm objects respectively are asynchronous. If they are not, you would need to modify those methods as well to be asynchronous.

For more details, you can refer to the Custom Query Engine () example on the LlamaIndex GitHub page.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

Add a reply

Find answers from the community

how should I async query>