METHOD AND APPARATUS FOR ESTIMATING A COMPLETION TIME FOR MAPREDUCE JOBS

摘要

A method, non-transitory computer readable medium, and apparatus for estimating a completion time for a MapReduce job are disclosed. For example, the method builds a general MapReduce performance model, computes one or more performance characteristics of each one of one or more benchmark workloads, computes one or more performance characteristics of the MapReduce job in the known processing system, selects a subset of the one or more benchmark workloads that have similar performance characteristics as the one or more performance characteristics of the MapReduce job, targets a cluster of processing nodes in a distributed processing system, computes one or more performance characteristics of the subset of the one or more benchmark workloads in the cluster of processing nodes and estimates the completion time for the MapReduce job.