Find answers from the community

Updated 2 months ago

Hi Has anyone got VertexAI and

Hi! Has anyone got VertexAI and SQLStructStoreIndex to work together?
Using a PaLM model but it seems to just skip past the SQL Query generation stage and makes a wild guess at the answer.
Attachment
image.png
L
l
7 comments
Thats.... super weird lol Not sure what's going on there. I've been meaning to try palm but haven't had a chance yet
Super weird for sure! I'm gonna try making a custom prompt template and mess around with that, but really any other directions of investigation super appreciated.
If you are used to using a debugger, or adding extra print statements to the installed library, I'd be curious what the raw output of this step is https://github.com/jerryjliu/llama_index/blob/0cf7f9983b6ec0528a327e8bc0e64bf0321b73fc/llama_index/indices/struct_store/sql_query.py#L153

The _ variable also contains the full input to the LLM. Was it too long? Did the model just output a blank string? Very sus
@Logan M The _ var contains the prompt, which seems right. But the response_str comes back empty.
In the self._synthesize_response a couple lines later, the response string comes back with the final answer ...
Attachment
image.png
Super weird that the model wouldn't answer that prompt 🫠

I wonder if you copy/paste the full prompt and call it with the raw vertex api, if it also responds blank? If so, at least it gives a starting point to debug either the prompt or the api settings πŸ‘€
@Logan M I got it responding by adjusting the prompt template - made it simpler and removed the stop_token="\nSQLResult:"
Cant say I'm 100% on what is going on here, but it now responds with SQL and runs the query!
Thanks for all the help so far, this community is amazing.
Attachment
image.png
Oh wow, super spooky. Yea thankfully the prompts aren't hard-coded haha glad, you got it working! :dotsCATJAM:
Add a reply
Sign up and join the conversation on Discord