The community member is building a backend with FastAPI and LlamaIndex, and is trying to understand how to handle multiple users at once using a chat engine. They are considering two options: 1) storing a dictionary of chat engines with chat IDs as keys, and retrieving the appropriate chat engine on each request, or 2) creating a SimpleChatStore to store the chat history for each interaction and initializing a chat engine with the history for each request.
The comments suggest that the best approach is to manage the chat history, either using a SimpleChatStore or some other manual method, as the only state that needs to be maintained is the chat history. The community members agree that this makes sense.
I'm building a backend (with fastapi) with llamaindex and trying to understand in general how to handle multiple users at once. I'm using chat_engine. I am not using websockets, but will have some kind of chat id for each interaction
What would be the best way to build this? Option 1: Store a dictionary of chat engines (with chat ids as keys) and then on each request, get the chat engine for that user Option 2: Create a SimpleChatStore and store the chat history for each interaction (by chat engine), and on each request initialize a chat engine with the history for that user.
I've looked through the documentation, and it still does not feel clear to me what is best practice
@Logan M gotcha, so storing the chat history, and on each request initiating a chat engine with that chat history (for that session id/conversation id or however it's being stored?)
