Hi all. Complete newbie here, spent the

At a glance

The community member is a newbie who has created a chatbot and wants to be able to store the chat memory so that users can continue their chat in a different session. They understand that LlamaIndex only keeps the memory within the same session, so they are looking for a recommended method to save the chat memory in a database and retrieve it later.

The comments suggest a few approaches:

You can get the chat history from chat_history = chat_engine.chat_history chat_json = json.dumps([x.dict() for x in chat_history]) # restoring from llama_index.llms import ChatMessage chat_history = [ChatMessage.parse_raw(x) for x in json.loads(chat_json)] # into chat engine chat_engine = index.as_chat_engine(..., chat_history=chat_history) # or directly into chat() response = chat_engine.chat("hello", chat_history=chat_history)

This approach works for a single run of the script, but the community member wants the history to persist between runs. The recommended approach seems to be saving the chat history to a database and restoring it later. The community member tried a simple approach of saving to and restoring from a MySQL database, and another community member confirmed that

ffarez

Hi all. Complete newbie here, spent the whole day looking for an answer, so I hope someone can help.

I've been able to create a chatbot, but now I want to be able to store the chat memory somewhere so that the user can continue their chat later in a different session.

As I understand it, LlamaIndex only keeps the memory within the same session. If so, what's the recommended method for saving the chat memory in a database and retrieving it later during another chat session?

Thank you!

10 comments

LLogan M

You can get the chat history from

Plain Text

chat_history = chat_engine.chat_history
chat_json = json.dumps([x.dict() for x in chat_history])

# restoring
from llama_index.llms import ChatMessage
chat_history = [ChatMessage.parse_raw(x) for x in json.loads(chat_json)]

# into chat engine
chat_engine = index.as_chat_engine(..., chat_history=chat_history)

# or directly into chat()
response = chat_engine.chat("hello", chat_history=chat_history)

ffarez

Thanks @Logan M I'll give this a try.

ffarez

Hi again. Ok so that works, but only during a single run of the script.

If I wanted the history to persist between running the script, what's the recommended approach? E.g. saving it to a database.

This is for a "chat with data" SaaS that will be used by different users, and each user will have its own chat sessions and chat history.

(Oh and I'm also new to Python!)

Thanks!

ffarez

Went for a simple approach of saving to and restoring from a mysql db. Does this look right for restoring the chat history?

ffarez

Attachment

ffarez

Attachment

LLogan M

That looks right to me! Although you miiiight also need to restore additional_kwargs in the ChatMessage, in addition to role and content

ffarez

Ah ok, thanks @Logan M ! What are the additional_kwargs for and where should I restore them? Sorry, still all new to this and can't find the documentation for it.

LLogan M

Additional kwargs are just extra info that openai (or other LLM providers) attach to messages (usually this is for function/tool calls)

You would restore it ideally in your loop above

Plain Text

for row in records:
  chat_history.append(
    ChatMessage(
      role=row[3],
      content=row[2],
      additional_kwargs=row[??]
    )
  )

Personally though, I would store an entire JSON of the chat message and restore from that directly

Plain Text

message_json = message.json()

[store in db]

for row in records:
  chat_history.append(
    ChatMessage.parse_raw(row[<json location>])
  )

ffarez

Thanks @Logan M . That's really helpful!

Add a reply

Find answers from the community

Hi all. Complete newbie here, spent the