LlamaIndex
Log in

Find answers from the community

Home
Members
Vicent W.
V
Vicent W.
Offline, last seen 2 months ago
Joined September 25, 2024
VVicent W.
·

MistralAI has just released `Mistral-7B-Instruct-v0.3`, a local LLM is capable of function

MistralAI has just released Mistral-7B-Instruct-v0.3, a local LLM is capable of function-calling. How do I use that capability with LlamaIndex, preferrably interfacing via Ollama?
8 comments
k
V
L
VVicent W.
·

Functions that I would like to wrap into

Functions that I would like to wrap into FunctionTools tend to exceed the context window of my locally-running LLMs. (I'm looking at you, Wikipedia tool.)

While I can wrap each tool with a "please summarize this" decorator function individually, I wonder if LlamaIndex has already a mechanism for bulk-wrapping all tools for a given (ReAct)Agent?
12 comments
L
V
VVicent W.
·

Pytest collects zero test cases. Am I

Pytest collects zero test cases. Am I missing something? 🧵
57 comments
V
L
S
VVicent W.
·

I noticed that my Agent has been using

I noticed that my Agent has been using the same tool in the same way repeatedly in one single chain-of-thought, probably due to the LLM being feeble and its context window being small.

How do implement a sort of "memoization" feature such that when I detect that this happens, instead of calling the tool, the code tells the LLM (perhaps as an appended System Message) saying that "you've tried this no long ago. Try something else."?

I can do this comfortably in vanilla Python (by implementing a decorator that wraps around the Python functions that I called to_tool for), but I wonder if there's a more conventional way to do this, esp. with the 0.10.x overhaul?
1 comment
V
VVicent W.
·

Quick question: I'm trying to harden

Quick question: I'm trying to harden this thing:
Plain Text
  File "/Users/meow/Library/Caches/pypoetry/virtualenvs/my-agent-0qUI9MIn-py3.10/lib/python3.10/site-packages/llama_index/core/agent/react/step.py", line 202, in _extract_reasoning_step
    raise ValueError(f"Could not parse output: {message_content}") from exc


so I did this to my pyproject.toml:
Plain Text
[tool.poetry.dependencies]
llama-index = { path = "../../llama_index", develop = true }

where ../../llama_index is where I cloned https://github.com/run-llama/llama_index.

but I'm still seeing the traceback saying .../site-packages/... other than the ../../llama_index path I'm expecting.
I usually was able to do this with pip install -e ../../llama_index, which I tried also and observed the same.

I think this might be due to the new file organization with >0.10.0? (I know I haven't been around for a while...)
2 comments
V
L
VVicent W.
·

**`self.state.get_task()` hangs when

self.state.get_task() hangs when called in a sub-thread

Hi all,

I'm working on the idea of giving Agents a sense of "back-burner", in the sense that it can "sleep on an idea" / "wait for the right time to come" for a certain task while responding to users' other queries.

The design: I kick off a separate thread that keeps a separate asyncio event loop spinning.

Plain Text
class DeferrableReActAgent(ReActAgent):
    def __init__(...) -> None:
        super().__init__(...)
        self.agent_worker = DeferrableReActAgentWorker.from_tools(...)
        self.deferral_loop = asyncio.new_event_loop()
        def __start_background_loop() -> None:
            asyncio.set_event_loop(self.deferral_loop)
            self.deferral_loop.run_forever()
        deferral_thread = Thread(target=__start_background_loop, daemon=True)
        deferral_thread.start()


I override the method _achat, such that -- when it deems that the it's not yet the right time to carry out a task -- it kicks off a clone of the same task in that sub-thread after 4 secs:
...
6 comments
V
k
VVicent W.
·

Memory

Hi all. I have an Agent. It tends to make multiple usages of various tools. When the chain of thought gets too long, the Agent tends to forget that it has already used a tool in the exact same way. I wish to summarize/compress the chain of thought so as to save some context window. Before I get started on implementing my own, I wonder if LlamaIndex already provides this functionality? Had no luck with the documentation or with Kapa AI. This is significant for locally-hosted LLMs, which tend to have shorter context windows. I’m using ReAct Agents, if that matters.
5 comments
h
L