session
object which is a dictionary that stores data across requests. When a client sends a request, Flask saves all the values stored in the session
object and sends them to the client. When the client sends another request, Flask restores the session
object with the values received from the client. However, Flask's session
is not suitable for storing large objects like your chat_engine
.ChatMemoryBuffer
and CondensePlusContextChatEngine
objects. You would need to make sure that these objects can be serialized and deserialized correctly, which might require some additional work.from flask import request, jsonify from llama_index.core.memory import ChatMemoryBuffer from llama_index.core.chat_engine import CondensePlusContextChatEngine from llama_index.core.message import ChatMessage import json @app.route('/chat', methods=['POST']) def chat(): # Get the conversation history from the request conversation_history_json = request.json.get('conversation_history') # Deserialize the conversation history conversation_history = [ChatMessage.from_dict(msg) for msg in json.loads(conversation_history_json)] # Create a chat memory buffer and load the conversation history memory = ChatMemoryBuffer.from_defaults(token_limit=3900) memory.set(conversation_history) # Create a chat engine chat_engine = CondensePlusContextChatEngine.from_defaults( index.as_retriever(), memory=memory, llm=llm, context_prompt=( "You are a chatbot, able to have normal interactions, as well as talk" " about the Kendrick and Drake beef." "Here are the relevant documents for the context:\n" "{{context_str}}" "\nInstruction: Use the previous chat history, or the context above, to interact and help the user." ), verbose=True, ) # Process the chat message message = request.json.get('message') response = chat_engine.chat(message) # Serialize the updated conversation history conversation_history_json = json.dumps([msg.to_dict() for msg in memory.get_all()]) # Send the response and the updated conversation history return jsonify({'response': str(response), 'conversation_history': conversation_history_json})
ChatMessage
object from your conversation history and store it in SimpleChatStore['chat_history']
. The ChatMessage
object requires a role
and content
. The role
can be either "system", "user", or "assistant". In your case, you should replace "bot" with "assistant". Here's how you can do it:from llama_index.core.message import ChatMessage from llama_index.core.storage.chat_store.simple_chat_store import SimpleChatStore # Your conversation history conversation_history = [ {'user': "How much is the cost to build a factory?"}, {'bot': "It is $500 to build a new factory"} ] # Convert the conversation history to a list of ChatMessage objects chat_messages = [] for message in conversation_history: for role, content in message.items(): if role == 'bot': role = 'assistant' chat_messages.append(ChatMessage(role=role, content=content)) # Create a SimpleChatStore and set the chat history chat_store = SimpleChatStore() chat_store.set_messages('chat_history', chat_messages)
ChatMessage
objects. We then create a SimpleChatStore
and use the set_messages
method to set the chat history.SimpleChatStore
can have implications for the performance and scalability of your application, so it's important to choose the right method for your needs.from llama_index.core.message import ChatMessage from llama_index.core.memory import ChatMemoryBuffer from llama_index.core.chat_engine import CondensePlusContextChatEngine # Your conversation history conversation_history = [ {'user': "How much is the cost to build a factory?"}, {'bot': "It is $500 to build a new factory"} ] # Convert the conversation history to a list of ChatMessage objects chat_messages = [] for message in conversation_history: for role, content in message.items(): if role == 'bot': role = 'assistant' chat_messages.append(ChatMessage(role=role, content=content)) # Create a chat memory buffer and load the conversation history memory = ChatMemoryBuffer.from_defaults(token_limit=3900) memory.set(chat_messages) # Create a chat engine self.chat_engine = CondensePlusContextChatEngine.from_defaults( index.as_retriever(), memory=memory, llm=llm, context_prompt=( "You are a chatbot, able to have normal interactions, as well as talk in a professional manner" "about the attached document(s)." "Here are the relevant documents for the context:\n" "{context_str}" "\nInstruction: Use the previous chat history, or the context above, to interact and help the user." ), verbose=False, ) return self.chat_engine
ChatMessage
objects. We then create a ChatMemoryBuffer
and use the set
method to load the conversation history. The ChatMemoryBuffer
is then passed to the CondensePlusContextChatEngine
.ChatMemoryBuffer
can have implications for the performance and scalability of your application, so it's important to choose the right method for your needs.ChatMessage
class is not located in llama_index.core.message
.ChatMessage
class is actually located in llama_index.core.llms
. So, you should be able to import the ChatMessage
class like this:from llama_index.core.llms import ChatMessage
llama_index
package is not installed or not installed correctly. You can install it using pip:pip install llama-index
llama_index
package and are still seeing the error, it could be an issue with your Python environment. Make sure that you're using the correct Python interpreter and that the llama_index
package is installed in the same environment.