Find answers from the community

Updated 4 months ago

Hi !

At a glance

The community member is trying to create a RAG (Retrieval-Augmented Generation) system using Mistral, a large language model. When using the "mistral-large" model, they encounter an error message: "Cannot stream response. Status: 400". This issue does not occur when using the "mistral-medium" model.

In the comments, another community member asks if the "streaming=False" argument is intentional. The original poster responds that it is not intentional, and that the error persists even when removing the "streaming" argument or setting it to "True".

There is no explicitly marked answer in the provided information.

Hi !
I try to create RAG system using Mistral.
When I use mistral-large, I get below error message.
raise MistralAPIException.from_response(mistralai.exceptions.MistralAPIException: Cannot stream response. Status: 400
This only occurs on mistral-large. When I use mistral-medium, this doesn't occur.

Code is below.
Plain Text
Settings.llm = MistralAI(model='mistral-large',
                    safe_mode=True, max_tokens=150)
query_engine = RetrieverQueryEngine.from_args(
            retriever=retriever,
            node_postprocessors=[
                MetadataReplacementPostProcessor(target_metadata_key="window")
            ],
            streaming=False,
            similarity_top_k=2,
            text_qa_template=QA_PROMPT,

        )
response = query_engine.query(query)
W
a
2 comments
You are passing streaming=False, is that intentional?
No, that isn't.
But, even though I remove "streaming" argument or change to "streaming==True", This error occurs.
Add a reply
Sign up and join the conversation on Discord