session object which is a dictionary that stores data across requests. When a client sends a request, Flask saves all the values stored in the session object and sends them to the client. When the client sends another request, Flask restores the session object with the values received from the client. However, Flask's session is not suitable for storing large objects like your chat_engine.ChatMemoryBuffer and CondensePlusContextChatEngine objects. You would need to make sure that these objects can be serialized and deserialized correctly, which might require some additional work.from flask import request, jsonify
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.chat_engine import CondensePlusContextChatEngine
from llama_index.core.message import ChatMessage
import json
@app.route('/chat', methods=['POST'])
def chat():
# Get the conversation history from the request
conversation_history_json = request.json.get('conversation_history')
# Deserialize the conversation history
conversation_history = [ChatMessage.from_dict(msg) for msg in json.loads(conversation_history_json)]
# Create a chat memory buffer and load the conversation history
memory = ChatMemoryBuffer.from_defaults(token_limit=3900)
memory.set(conversation_history)
# Create a chat engine
chat_engine = CondensePlusContextChatEngine.from_defaults(
index.as_retriever(),
memory=memory,
llm=llm,
context_prompt=(
"You are a chatbot, able to have normal interactions, as well as talk"
" about the Kendrick and Drake beef."
"Here are the relevant documents for the context:\n"
"{{context_str}}"
"\nInstruction: Use the previous chat history, or the context above, to interact and help the user."
),
verbose=True,
)
# Process the chat message
message = request.json.get('message')
response = chat_engine.chat(message)
# Serialize the updated conversation history
conversation_history_json = json.dumps([msg.to_dict() for msg in memory.get_all()])
# Send the response and the updated conversation history
return jsonify({'response': str(response), 'conversation_history': conversation_history_json})ChatMessage object from your conversation history and store it in SimpleChatStore['chat_history']. The ChatMessage object requires a role and content. The role can be either "system", "user", or "assistant". In your case, you should replace "bot" with "assistant". Here's how you can do it:from llama_index.core.message import ChatMessage
from llama_index.core.storage.chat_store.simple_chat_store import SimpleChatStore
# Your conversation history
conversation_history = [
{'user': "How much is the cost to build a factory?"},
{'bot': "It is $500 to build a new factory"}
]
# Convert the conversation history to a list of ChatMessage objects
chat_messages = []
for message in conversation_history:
for role, content in message.items():
if role == 'bot':
role = 'assistant'
chat_messages.append(ChatMessage(role=role, content=content))
# Create a SimpleChatStore and set the chat history
chat_store = SimpleChatStore()
chat_store.set_messages('chat_history', chat_messages)ChatMessage objects. We then create a SimpleChatStore and use the set_messages method to set the chat history.SimpleChatStore can have implications for the performance and scalability of your application, so it's important to choose the right method for your needs.from llama_index.core.message import ChatMessage
from llama_index.core.memory import ChatMemoryBuffer
from llama_index.core.chat_engine import CondensePlusContextChatEngine
# Your conversation history
conversation_history = [
{'user': "How much is the cost to build a factory?"},
{'bot': "It is $500 to build a new factory"}
]
# Convert the conversation history to a list of ChatMessage objects
chat_messages = []
for message in conversation_history:
for role, content in message.items():
if role == 'bot':
role = 'assistant'
chat_messages.append(ChatMessage(role=role, content=content))
# Create a chat memory buffer and load the conversation history
memory = ChatMemoryBuffer.from_defaults(token_limit=3900)
memory.set(chat_messages)
# Create a chat engine
self.chat_engine = CondensePlusContextChatEngine.from_defaults(
index.as_retriever(),
memory=memory,
llm=llm,
context_prompt=(
"You are a chatbot, able to have normal interactions, as well as talk in a professional manner"
"about the attached document(s)."
"Here are the relevant documents for the context:\n"
"{context_str}"
"\nInstruction: Use the previous chat history, or the context above, to interact and help the user."
),
verbose=False,
)
return self.chat_engineChatMessage objects. We then create a ChatMemoryBuffer and use the set method to load the conversation history. The ChatMemoryBuffer is then passed to the CondensePlusContextChatEngine.ChatMemoryBuffer can have implications for the performance and scalability of your application, so it's important to choose the right method for your needs.ChatMessage class is not located in llama_index.core.message.ChatMessage class is actually located in llama_index.core.llms. So, you should be able to import the ChatMessage class like this:from llama_index.core.llms import ChatMessage
llama_index package is not installed or not installed correctly. You can install it using pip:pip install llama-index
llama_index package and are still seeing the error, it could be an issue with your Python environment. Make sure that you're using the correct Python interpreter and that the llama_index package is installed in the same environment.