System prompt

MMaker

hi, please how do I specify a base prompt? I want to specify the chatGPT and give it some information on what it's name is, how to act etc. I found this example in the docs, but I assume this is not what I am looking for, since when I specify the context_str variable I get an error, so I guess there is another way to do this? Thanks https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_prompts.html

6 comments

LLogan M

I'm assuming you mean a sort of system prompt right?

Here's an example, specifc to chat models like gpt-3.5 and gpt 4. You'll want to set the text_qa_template and refine_template

Plain Text

from llama_index.prompts.prompts import QuestionAnswerPrompt, RefinePrompt
from langchain.prompts.chat import (
    AIMessagePromptTemplate,
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate
)

SYSTEM_PROMPT = SystemMessagePromptTemplate.from_template("Every response should be written like you are a pirate.")

CHAT_REFINE_PROMPT_TMPL_MSGS = [
    SYSTEM_PROMPT,
    HumanMessagePromptTemplate.from_template("{query_str}"),
    AIMessagePromptTemplate.from_template("{existing_answer}"),
    HumanMessagePromptTemplate.from_template(
        "We have the opportunity to refine the above answer "
        "(only if needed) with some more context below.\n"
        "------------\n"
        "{context_msg}\n"
        "------------\n"
        "Given the new context, refine the original answer to better "
        "answer the question. "
        "If the context isn't useful, output the original answer again.",
    ),
]


CHAT_REFINE_PROMPT_LC = ChatPromptTemplate.from_messages(CHAT_REFINE_PROMPT_TMPL_MSGS)
CHAT_REFINE_PROMPT = RefinePrompt.from_langchain_prompt(CHAT_REFINE_PROMPT_LC)

CHAT_QA_PROMPT_TMPL_MSGS = [
    SYSTEM_PROMPT,
    HumanMessagePromptTemplate.from_template(
         "Context information is below. \n"
        "---------------------\n"
        "{context_str}"
        "\n---------------------\n"
        "Given the context information and not prior knowledge, "
        "answer the question: {query_str}\n"
    )
]
CHAT_QA_PROMPT_LC = ChatPromptTemplate.from_messages(CHAT_QA_PROMPT_TMPL_MSGS)
CHAT_QA_PROMPT = QuestionAnswerPrompt.from_langchain_prompt(CHAT_QA_PROMPT_LC)

...

query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT, refine_template=CHAT_REFINE_PROMPT)

MMaker

yes thats it thanks! also I'm wondering how does the functionality of the code above differ from simply doing something like this instead?

Attachment

MMaker

i guess i found the answer, the difference is that your code allows the ChatGPT to tap into its general knowledge right?

LLogan M

Not quite. ChatGPT is setup to read 3 different types of messages: Human, AI, and System

System messages usually include rules for how the AI should act and think. Your example could have a similar effect, but I think using system prompts is just another dimension to prompt engineering 😅

MMaker

ahh i see thats interesting i did not know that, thank you!

Could you also please explain to me which part of the code is responsible for ChatGPT access to general knowledge vs only context knowledge, or how does that work? I had an issue where the bot only had access to context knowledge and I managed to fix it somehow, but if I am being frank, I am not entirely sure which part of the code had that effect 😅

LLogan M

General knowledge vs. Context is all controlled through prompts. Basically, you tell the model with specific instructions to use only the context provided (which is what llama index does)

But at the end of the day, it's up to the llm to follow those instructions properly

For example, the most commonly used prompts are the text qa prompt and refine prompt

https://github.com/jerryjliu/llama_index/blob/21782659c4c3e42654d4c6d62e754cc82f1ad9d1/llama_index/prompts/default_prompts.py#L90

And chatgpt has a specific refine prompt here
https://github.com/jerryjliu/llama_index/blob/21782659c4c3e42654d4c6d62e754cc82f1ad9d1/llama_index/prompts/chat_prompts.py#L12

Add a reply

Find answers from the community

System prompt