what would be the best way to use

We don't quite have a good integration for the pydantic program directly within an index query yet.

The closest integration is in an agent setting for choosing which index to query, something like https://github.com/jerryjliu/llama_index/blob/main/docs/examples/agent/openai_agent_with_query_engine.ipynb

Would be super open to a PR that adds some kind of deep integration though!

I might be able to give that a try tomorrow!

That would be amazing!

I'm undecided on the best way to integrate this, but If you are curious, it could either be a new query engine, or just a new response mode 🤔 Likely the query engine would be the easier approach though