BenchKit, a Tool for Massive Concurrent Benchmarking

摘要

Benchmarking numerous programs in a reasonable time requires the use of several (potentially multicore) computers. We experimented such a situation in the context of the MCC (Model Checking Contest @ Petri net) where we had to operate more than 52000 runs for the 2013 edition. This paper presents BenchKit, a tool to operate programs on sets of potentially parallel machines and to gather monitoring information like CPU or memory usage. It also samples such data over the execution time. BenchKit has been elaborated in the context of the MCC and will be used for the 2014 edition.