Hi, I've worked through the PG RAG

EEnzo

Hi, I've worked through the PG RAG tutorial which works fine but I'm having issues using chat engines -- it looks like context is never passed in to the chat_engine.

With the below:

Plain Text

    chat_engine = index.as_chat_engine(
        chat_mode="best",
        service_context=service_context,
    )
    print(chat_engine.chat("What did the author do growing up?"))

The chat_engine returns it doesn't know which author I'm talking about

This is the same with chat_mode="context"

Any advice?

14 comments

rrahul

If the state is not maintained correctly between calls to chat_engine.chat(), the engine might lose track of the context. Make sure that the conversation history is being updated and passed correctly with each interaction.

EEnzo

This happens on the first invocation -- there is no history at this point

EEnzo

But I might misunderstand

Here's some more code just for additional context (no pun intended) https://paste.ubuntu.com/p/kDJXtxCtyf/

LLogan M

Both best and context modes do not strictly use the index

best defaults to an agent. An agent looks at the chat history, and decides whether or not to use the query engine or respond directly

context retrieves context on every user message, and inserts it into the system prompt. The LLM can then respond using the context in the system prompt, or directly using the chat history

LLogan M

These modes are described here
https://docs.llamaindex.ai/en/stable/module_guides/deploying/chat_engines/usage_pattern.html#available-chat-modes

EEnzo

Ok I'm confused now 😄

The page above says for context

"context - Retrieve nodes from the index using every user message ..."

Is that not correct?

EEnzo

What would be a good chat mode to force retrieving context from the index?

EEnzo

Or am I doing it wrong and should be using an agent + "best" mode?

LLogan M

It is correct 😅 Isn't that what I also wrote above? context retrieves context on every user message...

LLogan M

The key here is it allows for normal conversation, by inserting those nodes into the system prompt, and then adding the rest of the chat history

EEnzo

Its says "from the index" -- thats why I was assuming it queries the index

LLogan M

index.as_retriever().retireve(...) is essentially what it does under the hood

LLogan M

If you want, you can force the agent to call the query engine at least once, using

Plain Text

from llama_index.tools import QueryEngineTool

tool = QueryEngineTool.from_defaults(
  index.as_query_engine(...),
  name="my_query_engine", 
  description="Usefor for querying information about X.", 
)

from llama_index import OpenAIAgent
agent = OpenAIAgent.from_tools([tool], verbose=False)

agent.chat("...", tool_choice="my_query_engine")

EEnzo

Thanks, I will try that

Add a reply

Find answers from the community

Hi, I've worked through the PG RAG