Find answers from the community

Updated 2 days ago

Can I have a workflow running indefinitely that manages other sub-workflows? Is that effic

@kapa.ai Can I have a workflow running indefinitely that manages other sub-workflows? Is that efficient? Or should I use a separate library or code or method to manage the workflows?
k
Z
L
6 comments
Please stand by @ZachHandley as I search the knowledge base. I normally take about 15 seconds or so. :wave:
----------------------
Based on the knowledge sources provided, I can offer some insights about workflows in LlamaIndex, particularly regarding nested workflows and their management. However, the information doesn't directly address running a workflow indefinitely or the efficiency of managing multiple sub-workflows.

LlamaIndex does support nested workflows, which allows you to create more complex and customizable workflow structures. As described in the Nested workflows documentation, you can create slots in existing workflows where you can supply additional workflows. This feature enables you to extend and customize workflows to suit your specific needs.

For example, you can create a main workflow that includes a slot for a nested workflow:

Plain Text
class MainWorkflow(Workflow):
    @step
    async def start(
        self, ctx: Context, ev: StartEvent, reflection_workflow: Workflow
    ) -> Step2Event:
        print("Need to run reflection")
        res = await reflection_workflow.run(query=ev.query)
        return Step2Event(query=res)


You can then supply a custom workflow to fill this slot:

Plain Text
w = MainWorkflow(timeout=10, verbose=False)
w.add_workflows(reflection_workflow=ReflectionFlow())
result = await w.run(query="Initial query")


While this demonstrates the ability to manage sub-workflows within a main workflow, the documentation doesn't mention running workflows indefinitely or provide information about the efficiency of managing multiple sub-workflows over extended periods.
Regarding whether you should use a separate library or method to manage workflows, the provided information doesn't offer a direct comparison or recommendation. LlamaIndex's workflow system seems designed for flexibility and extensibility within the context of LlamaIndex operations, but it may not be optimized for long-running, indefinite workflow management.

For a more definitive answer about running workflows indefinitely or managing large numbers of sub-workflows efficiently, you might need to consult additional LlamaIndex documentation or reach out to the LlamaIndex community for specific use case recommendations.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
@Logan M what would you recommend?
I don't need a FastAPI server, because that is unnecessary given the distance and it's not an API, it needs to be websocket-based, and support multiple sockets, and I basically want to create a workflow that is almost real-time, all function calling, and ideally can separate out concerns
You can definitely write a workflow that calls other workflows
Add a reply
Sign up and join the conversation on Discord