Modeling and simulation of distributed computing frameworks

摘要

A method receives a second data set that is different from a first data set. A total number of operations based on the second data set using an operation estimator is generated. Also, an aggregate resource cost for the total number of operations based on the second data set using a resource cost estimator is generated. The method generates a simulation driver file including a sequence of operations from the total number of operations and a resource cost for each operation in the sequence of operations from the aggregate resource cost. The method simulates the sequence of operations by performing: requesting an amount of resource used by a respective operation on the simulated distributed computing system; reserving the amount of resource when available in the simulated distributed computing system without executing the respective operation; and calculating a time period associated with a simulated execution time of the respective operation.