Method and apparatus for stateless parallel processing of tasks and work flows

摘要

A method is provided for parallel processing of a data stream. In this method, a processing task is received. The processing task is configured to process a data stream comprising a plurality of segments. A partitioning operation is subsequently performed on the data stream to partition the plurality of segments into N substreams, wherein each of the N substreams includes one or more segments of the plurality of segments, where N is a positive integer. . In addition, N subprocessing tasks are performed on the N substreams to generate N processed substreams. A merge operation is performed on the N processed substreams to produce a merged output data stream. The merged output data stream is identical to the output data stream produced when a processing task is applied directly to the data stream without a split operation.