It is not uncommon to evaluate the effectiveness of competing parallel processing scheduling, mapping, and allocation heuristics by applying a common set of randomly-generated task systems and comparing the performance of the resulting allocations in a statistical manner with one another. Although much research has been performed using this paradigm the authors believe that often the results of such experiments have been extrapolated beyond their range of applicability and provide little insight into determining the best heuristic for a given type of real-world problem. This paper presents evidence to support this assertion by analyzing the results of from the mathematical literature (i.e. the P-method and the Box method) to create a large set of directed graphs which are then used (along with a set of digraphs which were derived from real-world problems) to evaluate four classical list-based scheduling methodologies (the HLFET, HLFNET, SCFET, and SCFNET). The difference of the effective ranking of these methodologies from those presented by other researchers illustrate how the built-in biases associated with random techniques can affect how one views the relative effectiveness of each of these heuristics.
展开▼
