Find answers from the community

Updated 3 months ago

Hi, I've worked through the PG RAG

Hi, I've worked through the PG RAG tutorial which works fine but I'm having issues using chat engines -- it looks like context is never passed in to the chat_engine.

With the below:
Plain Text
    chat_engine = index.as_chat_engine(
        chat_mode="best",
        service_context=service_context,
    )
    print(chat_engine.chat("What did the author do growing up?"))

The chat_engine returns it doesn't know which author I'm talking about

This is the same with chat_mode="context"

Any advice?
r
E
L
14 comments
If the state is not maintained correctly between calls to chat_engine.chat(), the engine might lose track of the context. Make sure that the conversation history is being updated and passed correctly with each interaction.
This happens on the first invocation -- there is no history at this point
But I might misunderstand

Here's some more code just for additional context (no pun intended) https://paste.ubuntu.com/p/kDJXtxCtyf/
Both best and context modes do not strictly use the index

best defaults to an agent. An agent looks at the chat history, and decides whether or not to use the query engine or respond directly

context retrieves context on every user message, and inserts it into the system prompt. The LLM can then respond using the context in the system prompt, or directly using the chat history
Ok I'm confused now πŸ˜„

The page above says for context

"context - Retrieve nodes from the index using every user message ..."

Is that not correct?
What would be a good chat mode to force retrieving context from the index?
Or am I doing it wrong and should be using an agent + "best" mode?
It is correct πŸ˜… Isn't that what I also wrote above? context retrieves context on every user message...
The key here is it allows for normal conversation, by inserting those nodes into the system prompt, and then adding the rest of the chat history
Its says "from the index" -- thats why I was assuming it queries the index
index.as_retriever().retireve(...) is essentially what it does under the hood
If you want, you can force the agent to call the query engine at least once, using

Plain Text
from llama_index.tools import QueryEngineTool

tool = QueryEngineTool.from_defaults(
  index.as_query_engine(...),
  name="my_query_engine", 
  description="Usefor for querying information about X.", 
)

from llama_index import OpenAIAgent
agent = OpenAIAgent.from_tools([tool], verbose=False)

agent.chat("...", tool_choice="my_query_engine")
Thanks, I will try that
Add a reply
Sign up and join the conversation on Discord