Hi !
I try to create RAG system using Mistral.
When I use mistral-large, I get below error message.
raise MistralAPIException.from_response(mistralai.exceptions.MistralAPIException: Cannot stream response. Status: 400
This only occurs on mistral-large. When I use mistral-medium, this doesn't occur.
Code is below.
Settings.llm = MistralAI(model='mistral-large',
safe_mode=True, max_tokens=150)
query_engine = RetrieverQueryEngine.from_args(
retriever=retriever,
node_postprocessors=[
MetadataReplacementPostProcessor(target_metadata_key="window")
],
streaming=False,
similarity_top_k=2,
text_qa_template=QA_PROMPT,
)
response = query_engine.query(query)