I just want to pop in and complain real

Haha, sorry for all the 404s. And no its not becuase of 1st april.

Would you mind telling me what your were looking for. I'll try to help

Yeah I know it's not a 1st april thing, just want to air my frustration. Thanks for the help offer.
I am trying to do natural language query (text to sql) and need to do a few things but it seems there is a lack of documentations or I am just lacking.

I have been following
https://docs.llamaindex.ai/en/stable/understanding/putting_it_all_together/structured_data/
and https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/#part-2-query-time-retrieval-of-tables-for-text-to-sql

looking for a method to

1. Eliminate columns from the db or knowing if it's impossible so I can make a duplicate table of the data without the columns I don't want the query engine to query.
2. The default prompt for various models I think it's text-davicini-003? I had some luck with adding context_str but this can only slight nudge the query and not directly control it e.g format the respond in some way or exclude columns from being queried. Even a simple add this sentence in the end of the reponse is not being followed
3. The docs told me to use NLSQLTableQueryEngine but searching for more documentations leads to

This is not mentioned anywhere and causes confusion to say the least.

Thank you for the feedback.

1. I think column elimination should not be allowed ( IMO ) as it could lead to serious DB damage. Rest you can check with the queries like delete this column "column name" and check on the query that has been built to carry out the operation From the response object.

1. You can always modify the prompt to add/update the current prompt being used by your engine by following this: https://docs.llamaindex.ai/en/stable/examples/prompts/prompt_mixin/#accessing-prompts

1. You can replace NLSQLTableQueryEngine with SQLTableRetriever as it is deprecated.

Docs recently got revamped with new UI ( if you may have seen previous UI ). Some of the redirections may not be working, but most of them have been covered.

if you find any url, that throws 404, do share it. or Raise a PR 💪

Column elimination is a bad word to use I think.
It should be Column exclusion. Also the fact that the query_engine reads the schema directly from sqlalchemy is a source of potential problem too imo. When I first started I thought there was a way to load in defined schema e.g autogen schema following sqlalchemy's format
https://github.com/agronholm/sqlacodegen?tab=readme-ov-file
and modify that schema to specify which column has which mapping/description and only query from columns existing in the schema.

1. Thanks for the function.

I have been using this thread as it's related to what I am looking for but the default prompt links are 404. https://discord.com/channels/1059199217496772688/1059200010622873741/1092248540480290936

1. Thanks for the clarification. I don't remember the specific but I remember a few collab docs have old link leading to 404 resources (404 sample data/pdf files)

Overall if I have a big table with potentially sensitive data and only want the query engine to query a few safe columns the only way now is to duplicate the data to a safe table?

for default prompts: https://github.com/run-llama/llama_index/tree/main/llama-index-core/llama_index/core/prompts

Overall if I have a big table with potentially sensitive data and only want the query engine to query a few safe columns the only way now is to duplicate the data to a safe table?

I would prepare a table out of safe columns first as it will save you from injection queries as well.

Thank you for the detailed response.