Find answers from the community

Updated 3 months ago

I would find it very hard to believe

I would find it very hard to believe that I'm the first one to think of such a basic idea, but I'm looking for examples of open source projects and haven't come across any.

8 comments

LLogan M

I cant think of an example that does this, but I think theres two parts to this

how to decide when to store
how to decide when to read from the store

I think both usecases there could be solved just by making a couple of custom tools.

Downside is that it would increase latency due to the llm writing extra tool calls in between responses

bbin4ry_d3struct0r

The second part is a very interesting question. I'm thinking along the lines of simple prompt engineering (e.g., "Try and remember!") or key words/phrases along such lines to trigger search and retrieval.

I'll experiment with existing tools in LlamaIndex. I don't think it would be all that difficult to custom-build something.

LLogan M

Yea I kind of meant wraping the functionality into a function tool

Something like

Plain Text

def write_memory(memory: str) -> None:
  """Useful for recording a useful fact or memory to recall later."""
  ...

def read_memory(query: str) -> str:
  """Useful for looking up a previously recorded memory."""
  ...

write_tool = FunctionTool.from_defaults(write_memory)
read_tool = FunctionTool.from_defaults(read_memory)

bbin4ry_d3struct0r

Attachment

bbin4ry_d3struct0r

This is a slide from Jerry's talk at AIQCON a few months ago.

bbin4ry_d3struct0r

He mentioned the concept of a chat store for long-term storage of user chat history.

bbin4ry_d3struct0r

I'm now pondering the idea of using RAG to retrieve information relevant to the current conversation from this chat store instead of the data store (vector db).

LLogan M

The chat store is just a simple list of iteractions/messages the user has in a given chat session. You could definitely index this (and we also have a vector based chat store)

Add a reply