A community member is using OpenAIAgent, QueryEngineTool, and FunctionTool, but is facing an issue where the system returns results from outside the trained dataset, despite being trained on specific data. The community member would like to know if there are any options to limit the responses to only the data they have trained on.
Other community members suggest that this could be a prompt engineering problem, and recommend trying few-shot examples or asking @Logan M for specific techniques to stop the hallucinations. One community member provides an example where the chatbot has two indexes with mathematical formulas, and OpenAIAgent answers questions about these formulas efficiently, but still gives answers on other topics like literature, which is not desired. The community member asks if it is possible to prevent OpenAIAgent from using external data or provide keywords to use both indexes and FunctionTool.
There is no explicitly marked answer in the comments.
Hello. I am using OpenAIAgent along with QueryEngineTool and FunctionTool, but I am facing an issue where, despite training on specific data, it still returns results from outside the trained dataset. I would like to know if there are any options to limit responses to only the data I have trained on? Thanks all.
This looks like a prompt engineering problem. Try using few shot examples? Maybe @Logan M can help with some specific techniques to stop the hallucinations.
I'll give you an example: the chatbot has two indexes with many mathematical formulas taught, and OpenAIAgent answered my questions about these formulas efficiently. FunctionTool does a good job of solving some exercises. It still gives me answers when I ask about other topics, like literature, which is something I don't want to happen. I know OpenAIAgent's goal is to advance knowledge. Is it possible to prevent it from using external data or give me keywords so I can use both indexes and FunctionTool?
