task processing chains

tthoraxe

More of a pure langchain question but llamaindex may also have its own solution here.
trying to wrap my head around how to do something:
I'm submitting a question via RAG which gets turned into a list of tasks:

do a thing
do some other thing

for each step I am trying to:

figure out if the original question has enough information to complete the task
if it does, perform the task (via the LLM)
pass the original question, the output of the task, and the next step along
see if there is enough information to complete the next task

and kind of stay in that loop until everything is complete. then put itall together and send back to the user.

33 comments

tthoraxe

this smells vaguely of https://gpt-index.readthedocs.io/en/stable/examples/query_engine/recursive_retriever_agents.html

tthoraxe

or perhaps
https://gpt-index.readthedocs.io/en/stable/examples/query_transformations/SimpleIndexDemo-multistep.html

tthoraxe

i have very ugly python that kinda does 1-2 and part of 3 but the "continue and pass forward" part is where i'm falling down

LLogan M

This sounds very vaguely like a react agent loop 🙂

pass the original question, the output of the task, and the next step along what does this mean? You want to use this info in the next task? Or you want to keep this info until all tasks are complete?

tthoraxe

the current example is the following

tthoraxe

Original prompt: How do I configure my cluster for autoscaling up to a maximum of 10 nodes?

tthoraxe

we use RAG to pull a summary document, that document is sent to an LLM for further summarization, and it returns a task list:

Plain Text

    task_list = [
        "1. Determine the maximum number of nodes desired for the cluster.",
        "2. Create a ClusterAutoscaler that specifies the size of the cluster.",
        "3. Create a MachineAutoscaler object to specify which MachineSet should be scale and the minimum and maximum number of replicas.",
    ]

tthoraxe

we process the first task --
ask an LLM: is there enough information to do step 1?
oh, yes, there is? OK. actually do it.
Task Output: the maximum number of nodes is 10

tthoraxe

(or similar)

tthoraxe

then the next step would be "create a cluster autoscaler..."

tthoraxe

and we want to pass the original query plus the answer from step 1

tthoraxe

given the question 'how do I configure my cluster...' and the following information:
the maximum number of nodes is 10
do you have enough information to perform the task:
create a autoscaler...

tthoraxe

yes? ok, great. create an autoscaler given blahblah

tthoraxe

(interestingly, the create an autoscaler step needs to go to a different YAML generation model, but don't worry about that for now)

tthoraxe

current ugly python:
https://github.com/OpenShiftDemos/fastapi-lightspeed-service

tthoraxe

task_processor is where some of this logic lives

LLogan M

seems like once you have existing task outputs, you need to switch to a template that also includes the previous tasks+results?

LLogan M

The python looks pretty good btw, easy to understand

tthoraxe

thanks. it's my first "real" python project

tthoraxe

seems like once you have existing task outputs, you need to switch to a template that also includes the previous tasks+results?

maybe not the previous tasks, but definitely the results

tthoraxe

if we back up to less-specific details:

i get a question from a user
I look up in a document index for a relevant task summary
i iterate over the tasks, perform each task (if possible) and then feed all the state to the next task (in case it's relevant)
once all tasks are complete, return the whole hot mess to the user

tthoraxe

Q. How do I make such and such a sandwich?
RAG: this sandwich requires:
choose bread
gather ingredients
Place ingredients on bread
(some kind of processing loop)
self-Q: Can I figure out what bread is needed?
self-A: yes, it's wheat bread
self-Q: I need wheat bread, and can I figure out what other ingredients I need?
self-A: yes, it's a ham and cheese sandwich.

or something like that

tthoraxe

Does this make any sense?

LLogan M

Yea that makes total sense!

LLogan M

By the looks of it, you are nearly there. Just some minor adjustments to the processing loop and it's pretty much ready

tthoraxe

so you think that this really is a case for "more python"?

tthoraxe

(as opposed to some existing agent)

LLogan M

I think so? This kind of structured flow seems easiest to manage by creating your own chain of prompts to an LLM 🤔 Some existing frameworks might make this easier, but also it's simple enough you probably don't need more dependencies

tthoraxe

generally speaking if i find myself in a situation whre i need to write more code, i automatically expect that i am doing it wrong, and that someone beat me to it

tthoraxe

i'll do some more poking around

LLogan M

ha fair enough. There's a trade-off between writing it yourself vs. letting some library hide things from you.

In my personal opinion in this case, hiding the details seems like something that will make things more confusing. Since it's just a prompt + parsing an output to make a decision

LLogan M

dspy could maybe help with some bits, was watching the latest webinar for it

tthoraxe

will take a look, thanks

Add a reply

Find answers from the community

task processing chains