Pandora: An approach to analyzing safety-related digital-system failures.

摘要

Safety-related systems are those whose failure could result in loss of life, injury, or damage to property. The use of software and programmable electronic systems in safety-related domains, which include aerospace, commercial aviation, medicine, and nuclear power generation, is increasing. This increased reliance on digital systems to control potentially hazardous operations or to alert operators to dangerous conditions creates new failure modes and risks that might lead to accidents, and it poses new system development and safety assurance challenges.; Ensuring that digital systems will operate at least as dependably as the mechanical and analog systems they replace is essential, but achieving this level of dependability in a digital system can be exceptionally difficult. The design faults that plague digital systems are harder to identify and address than the physical faults that precede the bulk of mechanical and analog system failures. These design faults, coupled with the complex new designs that digital systems typically implement, complicate the safety assurance of digital systems. The increased reliance on digital systems to perform safety-related functions and the difficulty of ensuring that they will do so correctly increase the probability of accidents.; Analyzing safety-related failures of digital systems can yield lessons for improving development and assurance practices in order to reduce the risk of future accidents, but the same factors that complicate the safety assurance of these systems also affect failure analysis. Traditional techniques for investigating accidents assume that systems exhibit a common set of failure modes and that each failure mode leaves evidence that can be discovered from the accident scene. Such is not the case for digital systems, and so new techniques are needed to address the unique challenges that digital systems pose.; To address this problem, this dissertation introduces the Pandora approach to failure analysis. Pandora is a systematic but manual approach to analyzing safety-related failures of digital systems in which the analysis is framed around a system's safety case. The safety case documents the complete argument that the system is acceptably safe to operate, and framing failure analysis around the safety case provides important benefits. Investigators applying Pandora to a failure examine the safety case for fallacies; the presence of a fallacy in the safety case suggests the existence of a fault in the system that might have contributed to the failure. Pandora guides investigators through the steps of developing theories of the failure, eliciting evidence, and developing lessons and recommendations that address the problems the investigators identify. While Pandora may be applied to a wide array of system accidents, this dissertation focuses on its application to those involving safety-related digital systems.; Pandora is accompanied by a taxonomy of safety-argument fallacies to assist investigators in applying the process. The taxonomy documents fallacious reasoning that might appear in safety arguments and was developed from separate surveys of fallacies in real-world safety arguments and of fallacies documented in the philosophical literature. It may be used with Pandora or separately to assist in the detection of safety-argument fallacies.; Pandora was applied to a series of commercial-aviation accidents involving a minimum safe altitude warning system, and the safety-argument fallacy taxonomy was evaluated through a controlled study involving twenty computer-science graduate students, engineers, and safety professionals. In the former study, the application of Pandora produced findings comparable to those of the official investigations into the accidents. The latter study, while statistically inconclusive, suggests that the fallacy taxonomy assists the detection of fallacies in safety arguments. While both studies have significant limitations, they show that the Pand