On Testing Process Control Software for Reliability Assessment: the Effects of Correlation between Successive Failures

摘要

Statistical testing is the main available means for evaluating the reliability of software products. For ‘batch’ software, one can usually model both testing and operation as sequences of statistically independent trials (Bernoulli trials). Statistical inference from test results to reliability predictions is then straightforward. Things change when non-zero correlation is to be expected between the outcomes (failure or success) of successive executions of the software. This is the case, in particular, for most process control software: the inputs to each execution represent measurements on the controlled plant, and therefore follow quasi-continuous trajectories in the input space. For such software, this paper will: (i) show that the Bernoulli-trial model is inappropriate; (ii) argue that the ‘failure rate’ (probability of failure per execution) is no longer an appropriate indicator of software dependability; (iii) discuss the relationships among the statistical parameters describing the failure behaviour of the software; (iv) argue for direct measurement of the parameters of interest.