Find answers from the community

Updated 2 months ago

Functions that I would like to wrap into

Functions that I would like to wrap into FunctionTools tend to exceed the context window of my locally-running LLMs. (I'm looking at you, Wikipedia tool.)

While I can wrap each tool with a "please summarize this" decorator function individually, I wonder if LlamaIndex has already a mechanism for bulk-wrapping all tools for a given (ReAct)Agent?

12 comments

LLogan M

nothing built in 👀 I'm curious why the overflow? Too many tools? Tools have descriptions that are too long?

VVicent W.

I meant the response of the tools. e.g., in https://llamahub.ai/l/tools-wikipedia:

load_data: Loads a page from wikipedia

This is a heavy offender.

LLogan M

ah right! For that, there is an "on demand loader tool"

LLogan M

that creates a vector index on the fly

LLogan M

and searches it

LLogan M

This is a bad tutorial (it uses a langchain agent for some reason)
https://docs.llamaindex.ai/en/stable/examples/tools/OnDemandLoaderTool.html#ondemandloadertool-tutorial

LLogan M

But it gets the point across

VVicent W.

That's smart! Thanks for the pointer! (LOL it even uses Wikipedia loader already.)

VVicent W.

My initial idea was to append a TreeSummarizer to the output of the Wikipedia tool. IIUC, TreeSummarizer will retain information from broader raw output than OnDemandLoaderTool, because TreeSummarizer aggregates information layer-by-layer, while OnDemandLoaderTool selects only the top-k.
Just want to check if I'm making sense?

VVicent W.

Too many tools?

On a separate topic, I'm intrigued by this issue you brought up. What would be LlamaIndex's solution to "so many tools that the system prompt had taken >80% of the context window" problem? My guess is a router?

LLogan M

yea we have a concept of tool retrieval. So retrieve the top-k tools to answer each query. Not perfect but certainly helps 🙂

LLogan M

Yea that makes sense. Tree summarize will just be slower to run vs. top-k search 👍

Add a reply