Find answers from the community

Updated 3 months ago

I want to build a chatbot which can

I want to build a chatbot which can understand and answer user's questions.

there are 2 kinds of questions:
  1. pre-set standard questions. these are important, carefully selected, frequently asked questions in the past. we have carefully prepared standard answers to these questions(in the form of videos, articles, etc). when a user's question comes, the chatbot is expected to return the matched standard question(a following module will fetch the standard answer to the question and return to user). the chatbot should never try to answer a matched standard question by itself.
  2. other questions. if user's question doesn't match any of the standard questions, then the chatbot should fallback and try to answer it through gpt3.5.
I'm trying to customize a llama index chat engine to fulfil this. the chat engine consists of a query engine(which matches a query against standard question library), a llm(gpt3.5), and a system prompt. it can evaluate whether a question is a standard question or not, which is good. but the problem is, it will continue to forward the question to llm and return the final answer, which costs extra time and the result is useless. how can I stop this and let it just return the matched standard question when a match is found? attached is the code snippet(the full code is in https://github.com/pengxiaoo/llama-index-fastapi/blob/main/app/llama_index_server/index_server.py)
R
a
4 comments
Before calling the chat engine, you can compare user's query to the standard questions, and return the answer if matched.

Only call the chat engine if user's query doesn't match any of the standard questions.
this is a good suggestion. but in a conversational scenario, user's intention or meaning may "scatter" in multiple chat messages, so I think chat engine can help me to understand user's question by condensing the chat history. is there a way to get 2 birds in 1 stone? i.e. be able to match the standard questions and also get the benefit of chat history condensing from chat engine?
I see. I once faced similar situation, and it was easier for me to go with a custom option and to work with the condensed query. One option would be to compare the condensed standalone user query with the standard questions.

condense_question chat_mode might come useful in your scenario. Here's the docs: https://docs.llamaindex.ai/en/latest/examples/chat_engine/chat_engine_condense_question.html#
thanks @Rohan ! I will try
Add a reply
Sign up and join the conversation on Discord