Improving cybersecurity decision making by reducing expert bias: A review of expert knowledge elicitation methods.

摘要

This study identifies methods for eliciting knowledge from experts with minimal bias and evaluates their applicability to information security risk assessment, decision-making, and day-to-day operations. Decision makers rely on expert estimates in many fields, including information security. Research shows no consistent relationship between the estimation accuracy of experts and years of experience, publication record, or self-assessment as expertise. Critical infrastructure decisions are made based on estimates provided with stated 80% certainty or higher when those estimates in fact have 40--60% certainty. Researchers observed the effective application of bias reducing methods in many different fields. Questions and available data can be formatted in ways that ensure clarity and comprehension by experts. Calibration training can minimize under-confidence and over-confidence. Integrating estimates from multiple experts can improve accuracy and precision. Integrating data with expert estimates can also improve accuracy and precision of estimates. Simulation models can decrease bias, take into account irreducible uncertainty of the threat environment (variability), and allow analysts to calculate probabilities of highly complex scenarios. Simulation models can also be updated when new information becomes available and the threat and opportunities environment changes. The methods discussed in this capstone are applicable to high-level cybersecurity risk assessment and decision-making processes, as well as low-level technical SOC and CIRT daily operations.;Keywords: Cyber Security, Professor Albert Orbinati, risk, quantitative, value-based, assessment, methods, solutions.