Find answers from the community

Updated 2 months ago

Hi I've built a tool_indexer like below

Hi I've built a tool_indexer like below but it picks the wrong tool when i ask it to generate images.
Plain Text
tools_metadata = [
      {
        "query_engine": self.vector_query_engine,
        "metadata": ToolMetadata(
          name="user_uploaded_documents",
          description="use this tool when asked about specific documents uploaded by the user. Do not worry if the file does not mention the name of the file.",
        )
      },
      {
        "fn": generate_images,
        "description": (
          "generate_images(prompt: str, n: int) -> str\n\nOnly use this tool to create images or pictures upon user request. "
          "Useful for generating images"
        ),
        "fn_schema": DalleSchemaModel
      },
      {
        "fn": search_with_bing,
        "description": (
          "search_function(query: str) -> str\n\nUse this tool to retrieve real-time and up-to-date information to best answer a user query. "
          "This includes, but is not limited to, topics such as current events, weather updates, stock market data, and any other information that is subject to frequent changes"
        ),
        "fn_schema": BingSearchModel
      },
    ]
    for tool in tools_metadata:
      if "query_engine" in tool:
        tools.append(QueryEngineTool(query_engine=tool["query_engine"], metadata=tool["metadata"]))
      else:
        tools.append(FunctionTool.from_defaults(fn=tool["fn"], description=tool["description"], fn_schema=tool["fn_schema"]))

    tool_mapping = SimpleToolNodeMapping.from_objects(tools)
    tool_index = ObjectIndex.from_objects(
      tools,
      tool_mapping,
      VectorStoreIndex,
    )
      tool_retriever = tool_index.as_retriever(similarity_top_k=1)
      picked_tool = tool_retriever.retrieve(query)[0]
s
L
3 comments
my query is like can you create an logo image of a company selling t-shirts for gen-z
how does objectretriever retrieve work?
Its just using embeddings, comparing the embeddings of the query to the embeddings of each tool name+description
Add a reply
Sign up and join the conversation on Discord