Beacon-based distributed data processing platform

摘要

A method comprises initiating a first application in a first one of a plurality of distributed processing nodes, and responsive to initiation of the first application, identifying a plurality of beacon entities to be contacted in conjunction with execution of at least a portion of the first application. The method also comprises, for each of at least a subset of the identified beacon entities, initiating an additional application in an additional one of the plurality of distributed processing nodes. The method further comprises aggregating processing results from the first and one or more additional processing nodes, and providing the aggregated processing results to a client. The plurality of distributed processing nodes may comprise a plurality of YARN clusters associated with respective data zones, with each of the clusters being configured to perform processing operations utilizing local data resources locally accessible within its corresponding data zone.