LlamaIndex

Log inLog into community

Find answers from the community

Updated last year

Having an issue with azure openai.. but

Having an issue with azure openai.. but

At a glance

The community members are discussing an issue with the AzureOpenAI object, where it no longer has the _client attribute when running in a thread executor. They have upgraded the llama-index and openai packages, and the issue seems to be related to changes in the pickleability of the LLM, QueryEngine, and Index objects. The community members have worked on fixing the issue and have released a new version of the llama-index package, which they hope resolves the problem.

Useful resources

·

Having an issue with azure openai.. but only when running in a thread executor. Anyone else deal with this? Is the azureopenai object no longer thread safe? Was working before I upgraded llama index and openai packages

‘AzureOpenAI’ object has no attribute ‘_client’

L

b

28 comments

It should be even more thread safe than before 😅 no more using globals for azure

That's a super unexpected error... if you do pip show openai and pip show llama-index what do you get?

1.3.2 and 0.9.3

hmmmmmm

and you followed the latest docs? Some kwargs might have changed ever so slightly
https://docs.llamaindex.ai/en/stable/examples/customization/llms/AzureOpenAI.html

Yeah I had to change those last night to get it working again

So it works fine if I’m just running something standalone, but if I pass the object to asyncio event loop like loop.run_in_executor(llm…) that is when the error happens

ah I wonder if this is related to some changes I made to pickeling/deepcopy.... let me double check something

Also was looking at the azureopenai class and I couldn’t really follow the ‘_client’ instantiation 😅 I see the SyncAzureOpenAI but don’t know then it’s actually being called

ah crap it is related to those changes. Noooooo

Plain Text

>>> import pickle
>>> data = pickle.dumps(llm)
>>> new_llm = pickle.loads(data)
>>> new_llm.complete("Say one word! ")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/loganm/llama_index_proper/llama_index/llama_index/llms/base.py", line 313, in wrapped_llm_predict
    f_return_val = f(_self, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/loganm/llama_index_proper/llama_index/llama_index/llms/openai.py", line 203, in complete
    return complete_fn(prompt, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/loganm/llama_index_proper/llama_index/llama_index/llms/generic_utils.py", line 153, in wrapper
    chat_response = func(messages, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/loganm/llama_index_proper/llama_index/llama_index/llms/openai.py", line 238, in _chat
    response = self._client.chat.completions.create(
               ^^^^^^^^^^^^
AttributeError: 'OpenAI' object has no attribute '_client'. Did you mean: '_aclient'?

Ye that’s the exact traceback just async

Version

So I made indexs/query engines pickleable, but it seems like this destroys private variables

Let me see if I can fix this

and satisfy everyone

😆

Ah this makes sense

Way long ago I was trying to pass an index this way and was getting the non-pickle-able error

So I switched to just passing the llm

Ok, fixed for the LLM.... but now to check the query engine/index lol

huzzah, it seems to work! query engine, index, llm, and nodes/documents all pickle

need to head out for a flu shot lol but will push soon

Thanks!! Greatly appreciate it

Ok sorry for the delay

0.9.3.post1 is out

and I pray it works

😆

We are back in business, amazing, thank you!

Perfect!!

Add a reply

Sign up and join the conversation on Discord