Orchestration service for multi-step recipe composition with flexible, topology-aware, and massive parallel execution

摘要

Techniques are described for orchestrating execution of multi-step recipes. In an embodiment, a method comprises receiving a request to execute a recipe specification that defines a sequence of steps to execute for a particular recipe; responsive to receiving the request to execute the recipe specification, instantiating a set of one or more recipe-level processes; wherein each recipe-level process in the set of one or more recipe-level processes manages execution of a respective instance of the particular recipe; triggering, by each recipe-level process for the respective instance of the particular recipe managed by the recipe-level process, execution of the sequence of steps; wherein triggering execution of at least one step in the sequence of steps by a recipe-level process comprises instantiating, by the recipe-level process, a plurality of step-level processes to execute the step on a plurality of target resources in parallel.