Testing Concurrency Runtime via a Stochastic Stress Framework

摘要

The non-linear interaction of many software components makes quality assurance a complex problem even for traditional serial code. Concurrency and interaction from multiple threads add an additional dimension to software complexity. This extra dimension introduces unique bug types such as deadlocks, live-locks, and race conditions. In this paper, we will describe how a solid stress framework, complete with integrated structured randomization and methodical meddling of temporal properties makes practical software quality assurance possible. Specifically, we discuss the methods and practices applied to provide solid assurance to a critical commercial component - the native Concurrency Runtime [3] stack from Microsoft. First, by applying random distributions in individual tests and integrating such individual tests via a statistically fair scheduler, we describe how to cope with traversing the seemingly infinite interaction patterns. Second, we will expose how such testing helps identify hangs stemming from deadlocks and live-locks. Third, we will talk about injecting randomization to the temporal properties of the software system methodically, and how that can be used to assure that we find bugs with reasonable probabilistic expectation. We will conclude with a brief survey of the effectiveness of our stochastic stress framework over other tools.