Find answers from the community

Updated 4 months ago

I would find it very hard to believe

I would find it very hard to believe that I'm the first one to think of such a basic idea, but I'm looking for examples of open source projects and haven't come across any.
L
b
8 comments
I cant think of an example that does this, but I think theres two parts to this
  • how to decide when to store
  • how to decide when to read from the store
I think both usecases there could be solved just by making a couple of custom tools.

Downside is that it would increase latency due to the llm writing extra tool calls in between responses
The second part is a very interesting question. I'm thinking along the lines of simple prompt engineering (e.g., "Try and remember!") or key words/phrases along such lines to trigger search and retrieval.

I'll experiment with existing tools in LlamaIndex. I don't think it would be all that difficult to custom-build something.
Yea I kind of meant wraping the functionality into a function tool

Something like

Plain Text
def write_memory(memory: str) -> None:
  """Useful for recording a useful fact or memory to recall later."""
  ...

def read_memory(query: str) -> str:
  """Useful for looking up a previously recorded memory."""
  ...

write_tool = FunctionTool.from_defaults(write_memory)
read_tool = FunctionTool.from_defaults(read_memory)
This is a slide from Jerry's talk at AIQCON a few months ago.
He mentioned the concept of a chat store for long-term storage of user chat history.
I'm now pondering the idea of using RAG to retrieve information relevant to the current conversation from this chat store instead of the data store (vector db).
The chat store is just a simple list of iteractions/messages the user has in a given chat session. You could definitely index this (and we also have a vector based chat store)
Add a reply
Sign up and join the conversation on Discord