Async azure

At a glance

The community member is trying to test asynchronous functionality using the GPTSimpleVectorIndex.from_documents() and index.query() methods from the llama_index library. The first query runs fine, but the second query with the same index variable results in an APIError related to Azure OpenAI's content management policy.

Another community member suggests that the issue might be related to the first query still running when the second query is made, but they are unsure if that is the actual cause.

Useful resources

ssnapster

I'm trying to test async using this notebook https://github.com/jerryjliu/llama_index/blob/main/examples/async/AsyncQueryDemo.ipynb
When I run

Plain Text

index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
start_time = time.perf_counter()
index.query(query, use_async=True)
elapsed_time = time.perf_counter() - start_time

It runs fine. But if i do

Plain Text

start_time = time.perf_counter()
index.query(query, use_async=True)
elapsed_time = time.perf_counter() - start_time

with the same index variable i get this error

Plain Text

APIError: Invalid response object from API: '{"error":{"message":"The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400}}' (HTTP response code was 400)

Need some help @Logan M @jerryjliu0

1 comment

LLogan M

I have no clue on this one 🤔 I wonder if it has something to do with the first query still running when you make the second query? But that seems silly haha

Add a reply

Find answers from the community

Async azure