Systems, Methods, and Apparatuses for Implementing Concurrent Dataflow Execution with Write Conflict Protection Within a Cloud Based Computing Environment

摘要

In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing concurrent dataflow execution with write conflict protection within a cloud based computing environment. For instance, an exemplary system having at least a processor and a memory therein includes means for: creating a dataflow definition for a first dataflow type, wherein the dataflow definition includes at least one or more datasets to be accessed by the dataflow and a plurality of functional operations to be performed on the one or more datasets when the dataflow is executed; generating and storing a dataflow version identifying all datasets accessed by the dataflow based on the dataflow definition created; receiving multiple requests for the first dataflow type; enqueuing the multiple requests into a message queue pending execution; selecting, from the message queue, a first runnable dataflow having been earliest enqueued of the first dataflow type for execution based on (i) the first dataflow type being allowable within system limits and based further on (ii) verification that the selected first runnable dataflow is not already executing and based further on (iii) verification there is no write conflict for any dataset accessed by the selected first runnable dataflow. Other related embodiments are disclosed.