Find answers from the community

Updated 3 months ago

is there an option does not call openai

is there an option does not call openai api for text to sql?
L
h
7 comments
It just calls the normal LLM (text-davinci-003 by default)

However, it's only sending the table schema(s) plus the user queries, so not too data sensitive?

If that's still an issue though, yea you can use a local LLM, but you'll need powerful hardware to run it
Thanks Logan, I was thinking about this model. https://huggingface.co/juierror/text-to-sql-with-table-schema Is there anything in huggingface that you would suggest that would perform well for text-to-sql functionality?
Thinking about local LLM option because openai api has a quota for usage and because of cost.
That might be a good option, especially if you are only using a single table at a time
But to be honest, if all you need is the text-to-sql part, it might be easier to use that model directly, without llama-index in the mix
Thanks Logan, I tried out the huggingface text to sql and it does not work as well as the openai api method. The generated sql queries are wrong. Can you please point me to the place in the llama_index source code where you process the user input and call openai api for text to sql?
Add a reply
Sign up and join the conversation on Discord