Google vertex llm grounding tool configuration issue

At a glance

The community member is trying to use the LLM from Google Vertex IA through the llamaindex package, specifically the llama_index.llms.vertex.Vertex class. They are able to set up the model, but they are unable to figure out how to add a grounding tool to it. The community member notes that when using the model directly from the package provided by Google, they have the option to add a grounding tool, but they are unsure how to achieve the same from the llama_index library.

In the comments, another community member suggests that the integration of the grounding tool may not be available in the llama_index library, and they mention seeing a very old pull request for this feature that did not get merged.

VVitali

Hi I'm trying to use LLM from Google Vertex IA from llamaindex package from llama_index.llms.vertex import Vertex llm_vertex = Vertex(model="gemini-1.5-flash-002", additional_kwargs={}) , but I cannot figure out how to add grounding tool to that model. If I use this model from package provided by google , I have option to add grounding tool.

Plain Text

# search grounding tool for rag
search_ground = Tool.from_google_search_retrieval(grounding.GoogleSearchRetrieval())
llm_vertex = GenerativeModel(model_name="gemini-1.5-flash-002")
response = llm_vertex.generate_content(query, tools=[search_ground])

Any thoughts on how to achieve the same from the llama_index library?

2 comments

LLogan M

I don't think thats integrated

LLogan M

I might have saw a (very old) PR for this that didn't get merged for one reason or another

Add a reply

Find answers from the community

Google vertex llm grounding tool configuration issue