Chatbot context sharing issue across multiple users

Question

I am currently building a chatbot that reads data from different pdfs. I use chatengine to retrieve queries. chat engine keeps the context, but when multiple people are using the bot at the same time, the context is shared in the same memory.

I want every session to have individual memory, currently user 1 is asking about x, user 2 asking about y, then when user 1 is asking about tell me more, it says more about y instead of x because the last context it has is about y. It is not taking into account individual user’s context.
Can I please get some help on this @Logan M

Logan M · Answer

Separate the memory per-userYou can pass in the chat history to override at any timechat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objectsYou can get the current chat history with chat_engine.chat_history

Logan M · Answer

Feel free to manage and store that anywhere

Logan M · Answer

You could also use a chat storehttps://docs.llamaindex.ai/en/stable/module_guides/storing/chat_stores/

Pragyan Mohapatra · Answer

Ok I tried getting chathistory from chatengine.chat_history and use it inside in chatengine just the way you suggested.
chat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objects

It still takes latest chat context irrespective of the user

Pragyan Mohapatra · Answer

To answer the question tell me more

Pragyan Mohapatra · Answer

It is still giving more of latest question irrespective of the user

Logan M · Answer

Yes, but you need to manage that chat history per user (i.e. each user needs to have their own chat_history list)

Pragyan Mohapatra · Answer

@Logan M how do I do that

Logan M · Answer

Idk, its up to you lol its general software engineering at this point You could throw it into redis, some in-memory dictionary, some sql dbAnd just pull the chat history for each user as needed

Pragyan Mohapatra · Answer

Do we have unique user Id in the chat history @Logan M I see the list of ChatMessage Objects, how do we get the userids of the chatmessages, to understand which chatmessage is of whose userid

Logan M · Answer

Ideally only one user is chatting with a given agent/chat history at a given time right? Why would more than one user end up in the chat history of its being managed properly?

Pragyan Mohapatra · Answer

Many people can chat at the same time, isnt it? I want each of their memory retained to them separately.
When I pull the chat history to segregate, Can I get any I’d that specifies the unique user @Logan M

Logan M · Answer

I think you should maintain chat history per user

Logan M · Answer

Way easier

Logan M · Answer

I think I've suggested that earlier too 😅

Pragyan Mohapatra · Answer

Yes, but for that the chat history that comes back after chat is a list, so the last message should go to which user I’d if new users have started conversing @Logan M Is there any example?

Logan M · Answer

I think i gave an example right?It's quite simple response = chat_engine.chat("hello", chat_history=chat_history)You just have to maintain that list for each user. It's a list of pydantic objects so it's easy to serialize into any db or whatnot

Pragyan Mohapatra · Answer

I understand this part, But chat_history is coming for all the users @Logan M

Logan M · Answer

But it doesn't have to be?chat_histories = {"user1": [], "user2": []} response = chat_engine.chat("hello", chat_history=chat_histories["user2"]) chat_histories["user2"].append(ChatMessage(role="assistant", content=respons.response))

Logan M · Answer

If you are using an agent, you'll probably want to include the tool calls in the history. For that, I would just init the agent each timeagent = FunctionCallingAgent.from_tools(...)
response = agent.chat("hello", chat_history=chat_histories["user2"]) chat_histories["user2"] = agent.chat_history

Pragyan Mohapatra · Answer

Thanks @Logan M This worked. Also Can we get timestamp from ChatMessage

Pragyan Mohapatra · Answer

Since I have userids and corresponding chatmessage objects, I want to delete the userids that were used 1hour back

Pragyan Mohapatra · Answer

@Logan M

Logan M · Answer

You'll have to generate the timestamps yourself (this is pretty easy for for most SQL dbs, timestamps are built in)

Pragyan Mohapatra · Answer

Sure, Thanks a lot @Logan M Everything that you suggested worked.The only thing is response time is very large, how can I reduce it? Any suggestions? @Logan M

Find answers from the community

Chatbot context sharing issue across multiple users