COMPUTATION OF COMPOSITE FUNCTIONS IN A MAP-REDUCE FRAMEWORK

摘要

Embodiments of the present invention provide efficient systems and methods for processing large data sets using a composite function. Embodiments of the present invention can be used to compute a broad range of composite functions in a single map-reduce job. Each mapper computes an additive function G on a set of specified data partitions, and then passes the results to one or more reducers. The one or more reducers can then compute a function F, using the aggregate results of function G and data from a single partition.