When using pydantic classes with OpenAI

is it with OpenAIAgent?

Im using the query_engine on an index, using OpenAI API

I've followed this documentation:

https://docs.llamaindex.ai/en/stable/examples/query_engine/pydantic_query_engine.html#create-the-index-query-engine-openai

@ravitheja

Here's the full error log, but i think there are also some unrelated errors from Sentry in there.

It's just that sometimes it works, but sometimes it will throw this error, even happens with the same index.

@ravitheja any ideas? Or do you need some more context?

Not sure about why it is causing the error. @Logan M any idea on this?

Yea the best you can do is try/except. The LLM is not following instructions and outputting invalid JSON

At some point hopefully we make this more robust

Alright. Yea so i've added just a basic retry now. I'm either getting 'empty response' or the expected response. If it's an empty response i retry.

But not really ideal as it can still fail sometimes, even after 3 attempts... Right now I just try catch and then retry. Is there a better way of doing this?

Or anything else I can do to improve this?

@ravitheja @Logan M