Find answers from the community

Updated 2 months ago

Blaming llama_index/llama_index/program/...

Hello:
I have noticed that starting from v0.8.65 we are not using function_call but tool_choice instead. This breaks current Azure api call signature. it also does not respect the api version too. I can see the change here: https://github.com/run-llama/llama_index/blame/3f15ebf660758fda4611fb511b93cdb9513c595b/llama_index/program/openai_program.py#L169


Is there a specific way to use openai programs for Azure deployments?
K
L
R
36 comments
error is:
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Unrecognized request arguments supplied: tool_choice, tools', 'type': 'invalid_request_error', 'param': None, 'code': None}}
OpenAI updated their client, and function_call is not supported or used anymore it seems

You probably need to use the latest api_version from azure I'm guessing

I'm not sure what you mean by api version isn't respected πŸ€” Can you share your LLM setup?

I can also point you to the source code
https://github.com/run-llama/llama_index/blob/3f15ebf660758fda4611fb511b93cdb9513c595b/llama_index/llms/azure_openai.py#L122
Plain Text
AzureOpenAI(
    model="gpt-4",
    engine="<engine name>",
    azure_deployment="<engine name>",
    api_type="azure",
    api_key=api_key,
    azure_endpoint=<resource.openapi.azure...>,
    api_version="2023-08-01-preview",
    timeout=60,
)
Also, can you confirm this still happens on the latest version of llama-index?
same setup works for v0.8.64
anything newer is having the issue
Also I am not using PydanticProgram directly but over query engine.

Plain Text
cectorStoreIndexInstance.as_query_engine(
            service_context=service_context,
            response_mode=ResponseMode.COMPACT,
            text_qa_template=qa,
            refine_template=refine,
            verbose=True,
            output_cls=Klass,
Yea, thats around the time the new openai client was integrated

And you've tried 0.8.66 or newer? (sorry just confirming which version this is breaking for you)

If so, I'll throw this to a team member that has azure access
fails with 0.8.66 and 0.8.65. I think .66 is the newest
oh v0.8.67 43 minutes ago
hmm
let me check this one
I suspect 67 will also fail (66 had some fixes for pydantic programs, but I guess azure got missed)
Thanks for checking though! In the meantime, I'll notify the team πŸ™‚
yep thins one fails too
lib/python3.11/site-packages/openai/_base_client.py", line 877, in _request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'error': {'message': 'Unrecognized request arguments supplied: tool_choice, tools', 'type': 'invalid_request_error', 'param': None, 'code': None}} INFO: 127.0.0.1:57186 - "GET /session/b4d31e00-7a2d-41b3-90d3-befbddf7912f/pdf HTTP/1.1" 200 OK
Thank you πŸ™‚
Is there any way you can use a newer api_version ? Or is there a newer version to use?
Maybe azure hasn't quite caught up to these openai client changes :PSadge:
I have no issues of using latest version. but that one also failed
what is the latest, out of curiosity?
that's why I thought api version is not taken into account when passing kwargs around
2023-09-01-preview
hmmm I'm going to guess azure is still behind then. This client just came out a few days ago
Might have to wait for azure to catch up for tool calling
I think they will announce new GPT models in their DEV event next week.
Nice, and hopefully a new API version then πŸ™‚
but the thing I don't get is, we are the ones who are breaking the contract.

when I pass 2023-09-01-preview to client I expect client to conform its specs. but it is sending latest spec format anyways
Yea I agree. I think the AzureOpenAI client from the openai client should be converting this under the hood, but I guess they decided not to do that
I can see that client is successfully passing api version to the server.
but not populated required fields..
azure is low-key the worst to support haha

tbh I think you'll just have to downgrade the openai client and llama-index package until the new azure API is released
I did the same. staying at .64
not that cutting edge but it does the job.
looking forward for new azure models. or code adjustments.πŸ™‚ thanks a lot
Thanks for understanding! πŸ™
I encounter the same error, I've just PM'd you
Add a reply
Sign up and join the conversation on Discord