System Prompt

The system prompt kind of depends on which tool you are using.

For a chat engine, the system prompt is just a simple kwarg in the constructor

index.as_chat_engine(..., system_prompt="Talk like a pirate.")

ContextChatEngine.from_defaults(..., system_prompt="Talk like a pirate")

For a query engine, the system prompt is actually set in the service context

ServiceContext.from_defaults(..., system_prompt="Talk like a pirate.")

@Logan M Just a quick question. in my case I used a template and passing it as the template to chat.engine. Am I doing it correctly? or is there a better way to achieve this?

So the context chat engine doesn't use a text qa template

It takes your query, retrieves nodes with w
It, and puts the text from those nodes into the system prompt

You can provide additional instructions in the system prompt by specifying the system_prompt kwarg

index.as_chat_engine(..., system_prompt="Talk like a pirate.")

It might also make more sense if you see the code, if you are curious
https://github.com/jerryjliu/llama_index/blob/3c338ea59be0bc9b4b98bce3fdc6be895409852a/llama_index/chat_engine/context.py#L120

Is it because you are adding system prompt to each message?

Yea, the context engine prepends the system prompt every time

But if you just added as the system_prompt kwarg then it is just in the beginning of the conversation?

Yea, the system prompt is recorded and always inserted before answering each message 👍