The community member is having issues with their custom chat prompts. Previously, they set up a rule that if a user asks a non-business domain question, the chatbot should respond with "Please ask questions regarding the business domain." However, this prompt is not working as expected, and the chatbot is providing location information instead of the expected response. The community member has also tested other prompts, and they seem to not be processed correctly. The community members are seeking help to figure out what is going wrong.
In the comments, other community members suggest that the community member should set a system prompt, as the template they are using is only applied to the query engine, which is only used if the agent decides to call it. They also provide links to relevant documentation on building an agent with lower-level APIs and customizing the agent's behavior with a system prompt.
Hi guys. I'm having issues with my custom chat prompts. Previously, I set up a rule that if a user asks a non-business domain question, it should respond with 'Please ask questions regarding the business domain.' However, it's not working as expected. For instance, when I asked, 'Where is San Francisco located?' instead of the expected response, it provided a location.
This prompt used to work but stopped working last Thursday. I've also tested other prompts, and it seems like they aren't being processed correctly. Can someone help me figure out what's going wrong?
def construct_index(file_dir="./docs", index_dir="./data"): service_context = ServiceContext.from_defaults(llm=OpenAI(model="gpt-3.5-turbo")) documents = SimpleDirectoryReader(file_dir).load_data() index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context) index.storage_context.persist(persist_dir=os.path.join(index_dir, "index")) return index
index_stored = construct_index()
chat_text_qa_msgs = [ ChatMessage( role=MessageRole.SYSTEM, content="You are an expert Business Professional chatbot. ensure that users' questions are directly related to the business domain. If users ask a question outside the business domain, you will reply with, 'Please ask questions regarding the business domain.' " ), ChatMessage( role=MessageRole.USER, content="Context information is below.\n" "{context_str}\n" "Answer the question: {query_str}\n"),] text_qa_template = ChatPromptTemplate(chat_text_qa_msgs)
Yea tha example is just applied to query engines. But you are using a chat engine.
Think of it like two layers -- the agent on top, and the query engine underneath.
The agent looks at the chat history and newest message, and decides if it needs to use a tool (I.e. the query engine)
If so, it writes a query to that query engine, passes it in, and reads the response from the query engine, and then answers the users original message
So the prompt template you are passing in is only used for the query engine. The system prompt example I gave above is applied to the top level agent to control its behavior
