Hello, I would like to ask for some help

DDat Tran

Hello, I would like to ask for some help on updating prompt
https://docs.llamaindex.ai/en/stable/examples/prompts/prompt_mixin/#put-into-query-engine-get-response
according to this guide you can do query_engine.update_prompts() to update prompt but they use the high level query_engine = index.as_query_engine(response_mode="tree_summarize")
What I am using is

Plain Text

query_engine = SQLTableRetrieverQueryEngine(
    sql_database,
    obj_index.as_retriever(similarity_top_k=1),
    llm=function_llm,
)

and

Plain Text

query_engine.update_prompts(
    {"response_synthesizer:summary_template": qa_prompt_tmpl}
)

returns
ValueError: Module response_synthesizer not found.
What is the correct key for this update?

32 comments

WWhiteFang_Jr

You can check which prompts are being used by your engine currently with this:
https://docs.llamaindex.ai/en/stable/module_guides/models/prompts/usage_pattern/#accessing-prompts

Then update the prompt based on the list returned above.

DDat Tran

Thanks @WhiteFang_Jr but I encountered a weird problem with that approach
My keys are ['response_synthesis_prompt', 'sql_retriever:text_to_sql_prompt']
and the default prompt is this https://github.com/run-llama/llama_index/blob/520305c015aa57c4c38209dfa3d272d8d91b7a96/llama-index-core/llama_index/core/prompts/default_prompts.py#L188C1-L209C2
However if I literally copied the default prompt and update it the query will fail with this error "error": "'dialect'"
If I replace it prompt with something lighter like for example

Plain Text

template = (
    "We have provided context information below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given this information, please answer the question and each answer should start with code word AI Demos: {query_str}\n"
)

Attachment

DDat Tran

It will follow the query like so

Attachment

DDat Tran

That error is unsearchable so I am not sure what broke.

WWhiteFang_Jr

Is there more to this error? sort of traceback?

DDat Tran

that's all I got from the reponse
here's what I get from turning on debug. I skimmer over but there doesn't seem to be any identifiable information in there.

DDat Tran

The query looks correct so I am not sure what went wrong.

DDat Tran

The query I used

Plain Text

    "Given an input question, first create a syntactically correct {dialect} "
    "query to run, then look at the results of the query and return the answer. "
    "You can order the results by a relevant column to return the most "
    "interesting examples in the database.\n\n"
    "Never query for all the columns from a specific table, only ask for a "
    "few relevant columns given the question.\n\n"
    "Pay attention to use only the column names that you can see in the schema "
    "description. "
    "Be careful to not query for columns that do not exist. "
    "Pay attention to which column is in which table. "
    "Also, qualify column names with the table name when needed. "
    "You are required to use the following format, each taking one line:\n\n"
    "Question: Question here\n"
    "SQLQuery: SQL Query to run\n"
    "SQLResult: Result of the SQLQuery\n"
    "Answer: Final answer here\n\n"
    "Only use tables listed below.\n"
    "{schema}\n\n"
    "Question: {query_str}\n"
    "SQLQuery: "

DDat Tran

It just the default query but I updated it in

Plain Text

query_engine3.update_prompts(
    {"response_synthesis_prompt": qa_prompt_tmpl},
    # {"sql_retriever:text_to_sql_prompt": qa_template},
    # {"response_synthesizer:text_qa_template": qa_template},
)

DDat Tran

Hello again, can I have a pointer to this?

DDat Tran

@WhiteFang_Jr sorry for this

WWhiteFang_Jr

Hey! Sorry for the late reply.

One question: Does dialect was part of it earlier in the default prompt or you added it from your side?

DDat Tran

Apparently it's is there in the default prompt

DDat Tran

Attachment

DDat Tran

As you can see I reran with update prompt commented out, the printed out prompt from get prompt has dialect

DDat Tran

and it's exactly the same as the default prompt in github

DDat Tran

can you replicate this on your side?

WWhiteFang_Jr

Sure let me try
Could you share your code

DDat Tran

It's a bit messy, excuse me

DDat Tran

you can just mock the table to be whatever, the goal is to have the query engine do natural language query, with added context and modified query to return result conforming to a simple format

DDat Tran

@WhiteFang_Jr

WWhiteFang_Jr

Sure, taking a look in 10 mins

WWhiteFang_Jr

Yeah I'll use dummy tables

WWhiteFang_Jr

Hey

WWhiteFang_Jr

Just tried to follow: https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/duckdb_sql_query/?h=sqltableretrieverqueryengine#advanced-text-to-sql-with-our-sqltableretrieverqueryengine

WWhiteFang_Jr

And it is working fine

WWhiteFang_Jr

I'm getting response

WWhiteFang_Jr

You started facing error when you updated the prompt?

DDat Tran

yes

WWhiteFang_Jr

I am able to update the prompt schema

WWhiteFang_Jr

Without facing any issue:

Plain Text

text =( "Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. You can order the results by a relevant column to return the most interesting examples in the database."

"Never query for all the columns from a specific table, only ask for a few relevant columns given the question."
"ALWAYS FOLLOW THE GIVEN BELOW RULES."   # My change added here!
"Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Pay attention to which column is in which table. Also, qualify column names with the table name when needed. You are required to use the following format, each taking one line:"

"Question: Question here"
"SQLQuery: SQL Query to run"
"SQLResult: Result of the SQLQuery"
"Answer: Final answer here"

"Only use tables listed below."
"{schema}"

"Question: {query_str}"

"SQLQuery: ")

WWhiteFang_Jr

just followed this link

Add a reply

Find answers from the community

Hello, I would like to ask for some help