Thanks Logan And one more question what

Yea you are right, the table context goes into the text-to-sql prompt, along with the schema.

I've had the best luck using table context strings that just describe what each column in the table is.

So if I want to specify what user input map to which column, I should just put into text_to_sql prompt instead of table_context prompt right?

And table_context is mainly only used for general table description? But not handle the case like "If user mentioned amount, it means 'RevenueTotal' columnd" like this...

I think you are on the right track there yes 👍

You might also be interested in using something like GaurdRails to help keep SQL syntax correct:
https://shreyar.github.io/guardrails/examples/syntax_error_free_sql/

https://gpt-index.readthedocs.io/en/latest/how_to/output_parsing.html#guardrails

(I haven't had a chance to use this yet, but it might be helpful!)

Thanks Logan. It's gonna be really helpful! Thanks!

And one more question here: Where could I put table column schema, like if I only want to include some columns instead of all?

I put schema like this under text-to-sql prompt, but seems still they are choosing columns out of these. The reason why I'm trying to limit columns selection since sometimes the query choose wrong column.

The concern is that sometimes the query can run properly after adding text-to-sql prompt, but sometimes they can not. And I'm trying to understand if you have any suggestion regarding like table column limit? Or any other better suggestion?

Like where and how to put table limit if we have multiple tables, and table schema limit if we only want to select some column? And also if we have multiple tables, where to put JOIN condition as part of prompt?

Ngl, this is moving beyond my experience now 🤣

I do have a demo that uses multiple tables, and the LLM has no trouble generating queries that use join

If you want to exclude columns, I think that makes sense to exclude those columns from the schema text

Hi @Logan M , having question related to text_to_sql prmopt for calling GPTSqlStoreIndex of index.query(text_to_sql_prompt=prompt_template). For this Prompt template, how to configure to let index run only the SQL query defined from template instead of generating itself?

I'm not sure 🤔 do you mean the prompt template has the exact sql you want to run? Do you have an example of the use case?

Hi @Logan M Actually I have another question for QuestionAnswerPrompt for GPTSimpleVectorIndex. I want the response for index.query to return only based on prompt templated configured from questionAnswerPrompt, instead of generating based on it's own context str. I tried to put template, and also directly listed like this: "Please answer question only and exactly using return answers listed from above context str information: {query_str}\n"
"Not providing any extra information, descriptions or answers.\n". In most times, it can work. But sometimes, it still not follow the template but generate result without following template. How to make the response follow prompt template in most times properly?

You might also need to set the refine_template -- if the text is too big to fit into a single prompt, it will query across several prompts and switches to the refine prompt template

How to combine prompt_template with refine_template? What is the difference between them? Currently, I have almost 30 lines for prompt_template.

Here's the default refine template: https://github.com/jerryjliu/llama_index/blob/main/gpt_index/prompts/default_prompts.py#L90

They are used independently, in the same query.

You can define your own refine prompt similar to your custom QA prompt, and then pass it in like this:

index.query(..., text_qa_template=my_qa_template, refine_template=my_refine_template)

have you ever got a reply to this? we are 01/29/2024 and i just cant have a simple join in my prompt, its almost ridiculous