Hi Has anyone got VertexAI and

Thats.... super weird lol Not sure what's going on there. I've been meaning to try palm but haven't had a chance yet

Super weird for sure! I'm gonna try making a custom prompt template and mess around with that, but really any other directions of investigation super appreciated.

If you are used to using a debugger, or adding extra print statements to the installed library, I'd be curious what the raw output of this step is https://github.com/jerryjliu/llama_index/blob/0cf7f9983b6ec0528a327e8bc0e64bf0321b73fc/llama_index/indices/struct_store/sql_query.py#L153

The _ variable also contains the full input to the LLM. Was it too long? Did the model just output a blank string? Very sus

@Logan M The _ var contains the prompt, which seems right. But the response_str comes back empty.
In the self._synthesize_response a couple lines later, the response string comes back with the final answer ...

Super weird that the model wouldn't answer that prompt 🫠

I wonder if you copy/paste the full prompt and call it with the raw vertex api, if it also responds blank? If so, at least it gives a starting point to debug either the prompt or the api settings 👀

@Logan M I got it responding by adjusting the prompt template - made it simpler and removed the stop_token="\nSQLResult:"
Cant say I'm 100% on what is going on here, but it now responds with SQL and runs the query!
Thanks for all the help so far, this community is amazing.

Oh wow, super spooky. Yea thankfully the prompts aren't hard-coded haha glad, you got it working! :dotsCATJAM: