I m noticing the chatprompts dont have

ccmagorian

I'm noticing the chatprompts dont have much effect on the output from OpenAI. Where can we see what the string looks like when its being sent to OpenAI for completion? I guess I just want to understand where the the prompts come into play in the pipeline

20 comments

LLogan M

If you turn on debug logging, you'll see the full text

Plain Text

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

LLogan M

In addition to the query string, there is a prompt template that the query string and retrieved context get thrown into

LLogan M

cc @Itamar

ccmagorian

What do we need to pass to as_chat_engine? It seems that text_qa_template and refine_template kwargs dont work

ccmagorian

or have no effect

ccmagorian

This document just shows as_query_engine() https://gpt-index.readthedocs.io/en/latest/examples/customization/prompts/chat_prompts.html

LLogan M

which chat engine are you using? The default? They are all slightly different

ccmagorian

openai

ccmagorian

or i guess the default

LLogan M

So passing in text_qa_template and refine_template works, but it only applies to the actual query engine itself

For example, the flow is

User sends message
Agent decides which tool to use
If it picks a tool, it writes it's own input to that tool
In this case, it would call your query engine, where the templates are used
Then the agent sees the answer, decides if anymore tools should be used, and re-writes the tool ouput(s) using the chat history to answer the original user message

In this flow, the templates only get used if the agent decides to use a query engine tool

LLogan M

If you want to have some control over the agent behaviour, you can pass in a system_prompt string

index.as_chat_engine(..., system_prompt="Talk like a pirate")

ccmagorian

Aha! That worked

ccmagorian

Can we combine Prompts with the chat engine by forcing the chat engine to use a query tool?

LLogan M

yea you can force one tool to at least be called every time

Plain Text

index.as_chat_engine(..., function_call="query_engine_tool")

ccmagorian

im assuming that the string "query_engine_tool" is a variable that isinstance QueryEngineTool from the index?

LLogan M

It's actually the name of the query engine tool, the string there is the default name 🙂

ccmagorian

how does that relate to a QueryEngineTool that we define?

ccmagorian

or do we not need to define one?

LLogan M

When you do index.as_chat_engine() it defines one for you

If you do OpenAIAgent.from_tools(), then you will have already defined the tool manually, and hopefully given that tool a name+description 🙂

ccmagorian

aha i see

Add a reply

Find answers from the community

I m noticing the chatprompts dont have