Find answers from the community

Home
Members
CryptRillionaire.eth
C
CryptRillionaire.eth
Offline, last seen 3 months ago
Joined September 25, 2024
Seems like a lot of the classes can be used for the same thing another class is supposed to be useful for and these a kinda overloading of operations.... Like for example it seems that a QueryRouterEngine can also just act exactly as an Agent. That is because it kinda is an agent with maybe some exatra tooling? Seems to make things kinda hard to reason about without really getting into the code of the API.
1 comment
L
I have an OpenAIAgent and im trying to feed it a list of messages. Everything method im finding takes a string to append to the messages. .*chat(str, history). Im just wanting to call completion on the list of messages. popping off the last message and submitting it as a string im sure would be incorrect usage of the library and im sure im missing something. Is working With the Agent really what I want? Do i want something else in the stack like the chat_engine instance?
11 comments
C
L
g
Could anyone explain what a LlamaIndex Agent expects in an astream_chat() call when it receives tools to use? Specifically, in a non-streaming call, I send back a list of tools parsed from the LLM response in the request.tools list.

I'm trying to determine whether, on my backend, I should:
  1. Detect a tool call, stop yielding tokens to the agent until I encounter a stop token, then parse out the tools and send the final buffered request with request.tools included,
or

  1. Continue streaming tokens to the agent and only in the final response send the request back with request.tools.
Which approach is correct?
10 comments
C
L
Has anyone been able to get a grammar file working in the Llama-cpp?
3 comments
D
C
Can anyone tell me Why this agent does not even attempt to call the tool parsing functions when it gets the perfectly formatted response form the LLM? Ive been digging through the sourcecode for hours now....

Plain Text
def add(x: int, y:int) -> int:
    """
    Add two numbers together
    """
    return x + y

add_tool = FunctionTool.from_defaults(fn=add)

tools: List[BaseTool] = [add_tool, subtract_tool, multiply_tool]

llm = OpenAILike(
    logprobs=None,
    api_version="v2",
    model="Llama-3.1-8b",  # Replace with your model's name
    api_base=API_BASE,
    api_key=API_KEY,
    max_tokens=100,
    is_chat_model=True,
    is_function_calling_model=True,
)

agent = OpenAIAgent.from_tools(
    system_prompt=system_prompt,
    chat_history=messages,
    tools=tools,
    llm=llm,
    verbose=True,
    tool_call_parser=advanced_tool_call_parser,
    is_function_calling_model=True,
    allow_parallel_tool_calls=True,
    is_chat_model=True,
)


res: str = agent.chat("Can you add together 5 + 9 for me.")
16 comments
C
L
J