Hi !

At a glance

The community member is trying to create a RAG (Retrieval-Augmented Generation) system using Mistral, a large language model. When using the "mistral-large" model, they encounter an error message: "Cannot stream response. Status: 400". This issue does not occur when using the "mistral-medium" model.

In the comments, another community member asks if the "streaming=False" argument is intentional. The original poster responds that it is not intentional, and that the error persists even when removing the "streaming" argument or setting it to "True".

There is no explicitly marked answer in the provided information.

aaiua

Hi !
I try to create RAG system using Mistral.
When I use mistral-large, I get below error message.
raise MistralAPIException.from_response(mistralai.exceptions.MistralAPIException: Cannot stream response. Status: 400
This only occurs on mistral-large. When I use mistral-medium, this doesn't occur.

Code is below.

Plain Text

Settings.llm = MistralAI(model='mistral-large',
                    safe_mode=True, max_tokens=150)
query_engine = RetrieverQueryEngine.from_args(
            retriever=retriever,
            node_postprocessors=[
                MetadataReplacementPostProcessor(target_metadata_key="window")
            ],
            streaming=False,
            similarity_top_k=2,
            text_qa_template=QA_PROMPT,

        )
response = query_engine.query(query)

2 comments

WWhiteFang_Jr

You are passing streaming=False, is that intentional?

aaiua

No, that isn't.
But, even though I remove "streaming" argument or change to "streaming==True", This error occurs.

Add a reply

Find answers from the community

Hi !