Logan M Few questions about prompts in

Few questions about prompts in query engines and agents:

text_qa_template - if I'm using a openai chat model, can I use ChatPromptTemplate.format_messages as input to this? I see examples just using a single huge string with the prompt, with context_str and query_str variables - I would like to use system, and user messages with custom variables (that I format and add before I pass the formatted prompt messages to text_qa_template when creating a query engine and its tool) apart from the context and query variables.
How do I add prompts to agents? I see there is prefix_messages and system_prompt - not sure which to use - does prefix_messages basically get added before every call?

15 comments

Yea exactly. There's actually an example of this here: https://gpt-index.readthedocs.io/en/stable/examples/customization/prompts/chat_prompts.html
You got it. Prefix messages will get added to every call. Typically this is for a system prompt, but could be anything. It will be added to the top of the chat history

So for 2, how does it get added to chat history - does it get added with every message, or only once at the beginning of chat history?

VVish

In terms of execution, it looks like it gets added to every query

LLogan M

Yea exactly

VVish

added with every message to chat history?

VVish

For 1, it looks like I can't add custom variables to the text_qa_prompt and only format it for the custom variables and leave context_str, and query_str intact before I add it to my query_engine init call

LLogan M

agent.chat("message") -> [prefix_messages, chat_history, "message"]

agent.query("message") -> [prefix_messages, "message"]

LLogan M

yea thats possible

VVish

How is that possible?

LLogan M

prompt.partial_format

VVish

Ohhh

VVish

Thanks for that