Find answers from the community

Updated 2 months ago

Example notebook for parallel function calling in workflow

Hey all, is there an example notebook where parallel function calling is being used in a workflow?
L
D
6 comments
Probably depends on what you mean by "parallel function calling"

If functions are properly implemented with async, you can easily execute them concurrently

If they aren't, you can just do asyncio.to_thread(..) to make it async, and then execute them in an asyncio.gather() call
In this case I actually just mean setting allow_parallel_tool_calls = True for a function calling LLM, curious how that works alongside parallel execution via increasing the num_workers in a workflow step and gathering tool calls
Here I'm looking at Anthropic specifically
yea, for example if you have a step for specifically calling tools, you can emit X number of events to trigger that step, and num_workers will control how many can run at a given time
The caveat here is that this is async, not multiprocessing, so if something blocks the event loop, it will run sequentially
(hence the tip to use asyncio.to_thread if your tools aren't async)
Add a reply
Sign up and join the conversation on Discord