Find answers from the community

Updated last month

Chatbot context sharing issue across multiple users

I am currently building a chatbot that reads data from different pdfs. I use chatengine to retrieve queries. chat engine keeps the context, but when multiple people are using the bot at the same time, the context is shared in the same memory.

I want every session to have individual memory, currently user 1 is asking about x, user 2 asking about y, then when user 1 is asking about tell me more, it says more about y instead of x because the last context it has is about y. It is not taking into account individual user’s context.
Can I please get some help on this @Logan M
L
P
25 comments
Separate the memory per-user

You can pass in the chat history to override at any time

chat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objects

You can get the current chat history with chat_engine.chat_history
Feel free to manage and store that anywhere
Ok I tried getting chathistory from chatengine.chat_history and use it inside in chatengine just the way you suggested.
chat_engine.chat(..., chat_history=chat_history) -- where chat history is a list of chat message objects

It still takes latest chat context irrespective of the user
To answer the question tell me more
It is still giving more of latest question irrespective of the user
Yes, but you need to manage that chat history per user (i.e. each user needs to have their own chat_history list)
@Logan M how do I do that
Idk, its up to you lol its general software engineering at this point

You could throw it into redis, some in-memory dictionary, some sql db

And just pull the chat history for each user as needed
Do we have unique user Id in the chat history @Logan M I see the list of ChatMessage Objects, how do we get the userids of the chatmessages, to understand which chatmessage is of whose userid
Ideally only one user is chatting with a given agent/chat history at a given time right? Why would more than one user end up in the chat history of its being managed properly?
Many people can chat at the same time, isnt it? I want each of their memory retained to them separately.
When I pull the chat history to segregate, Can I get any I’d that specifies the unique user @Logan M
I think you should maintain chat history per user
I think I've suggested that earlier too 😅
Yes, but for that the chat history that comes back after chat is a list, so the last message should go to which user I’d if new users have started conversing @Logan M
Is there any example?
I think i gave an example right?

It's quite simple

Plain Text
response = chat_engine.chat("hello", chat_history=chat_history)


You just have to maintain that list for each user. It's a list of pydantic objects so it's easy to serialize into any db or whatnot
I understand this part, But chat_history is coming for all the users @Logan M
But it doesn't have to be?

Plain Text
chat_histories = {"user1": [], "user2": []}

response = chat_engine.chat("hello", chat_history=chat_histories["user2"])

chat_histories["user2"].append(ChatMessage(role="assistant", content=respons.response))
If you are using an agent, you'll probably want to include the tool calls in the history. For that, I would just init the agent each time

Plain Text
agent = FunctionCallingAgent.from_tools(...)
response = agent.chat("hello", chat_history=chat_histories["user2"])

chat_histories["user2"] = agent.chat_history 
Thanks @Logan M This worked. Also Can we get timestamp from ChatMessage
Since I have userids and corresponding chatmessage objects, I want to delete the userids that were used 1hour back
You'll have to generate the timestamps yourself (this is pretty easy for for most SQL dbs, timestamps are built in)
Sure, Thanks a lot @Logan M
Everything that you suggested worked.

The only thing is response time is very large, how can I reduce it? Any suggestions? @Logan M
Add a reply
Sign up and join the conversation on Discord