Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
😞
😐
😃
Powered by
Hall
Inactive
Updated 3 months ago
0
Follow
Hello,
Hello,
Inactive
0
Follow
T
Thomas1234
8 months ago
·
Hello,
Is anyone getting this type error when using mistral API for async streaming ?
TypeError: object async_generator can't be used in 'await' expression
L
T
7 comments
Share
Open in Discord
L
Logan M
8 months ago
full traceback?
T
Thomas1234
8 months ago
{
"name": "TypeError",
"message": "object async_generator can't be used in 'await' expression",
"stack": "---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[13], line 4
1 from llama_index.core.llms import ChatMessage, MessageRole
3 message = ChatMessage(role=MessageRole.USER, content="Tell me a joke.")
----> 4 resp = await llm_c.astream_chat([message])
5 async for r in resp:
6 print(r.delta, end="")
File c:\Users\\anaconda3\envs\llama\lib\site-packages\llama_index\core\llms\callbacks.py:71, in llm_chat_callback.<locals>.wrap.<locals>.wrapped_async_llm_chat(_self, messages,
kwargs
)
54 dispatcher
.event
(
55 LLMChatStartEvent
(
56 model
_dict
=
_self
.to
_dict
(
)
,
(
.
.
.
)
60
)
61
)
62 event
_id
= callback
_manager
.on
_event
_start
(
63 CBEventType
.LLM
,
64 payload
=
{
(
.
.
.
)
68
}
,
69
)
---
> 71 f
_return
_val
= await f
(
_self
, messages
,
kwargs)
72 if isinstance(f_return_val, AsyncGenerator):
73 # intercept the generator and add a callback to the end
74 async def wrapped_gen() -> ChatResponseAsyncGen:
File c:\Users\\anaconda3\envs\llama\lib\site-packages\llama_index\llms\mistralai\base.py:327, in MistralAI.astream_chat(self, messages,
kwargs
)
324 messages
= to
_mistral
_chatmessage
(messages
)
325 all
_kwargs
= self
.
_get
_all
_kwargs
(
kwargs)
--> 327 response = await self._aclient.chat_stream(messages=messages, **all_kwargs)
329 async def gen() -> ChatResponseAsyncGen:
330 content = ""
TypeError: object async_generator can't be used in 'await' expression"
}
L
Logan M
8 months ago
ah, seems like a small bug I suppose, the method itself isn't async but it returns an async generator
T
Thomas1234
8 months ago
Oh right I see, do you want me to do a PR to fix it or create an issue on GitHub ?
L
Logan M
8 months ago
If you have time to make a PR, that would be great! Should be an easy change to make I think
T
Thomas1234
8 months ago
Ill see what I can do
Thanks 😊
T
Thomas1234
8 months ago
@Logan M found it,
Waiting for the review now but it’s not the biggest or you’ll see
Add a reply
Sign up and join the conversation on Discord
Join on Discord