Event induced bias in label fusion

摘要

In a two class label scenario, classification systems may be used to assess whether or not an element of interest belongs to the "target" or "non-target" class. The performance of the system is summarized visually as a tradeoff between the proportions of elements correctly labeled as "target" plotted against the proportion of elements incorrectly labeled as "target." These proportions are empirical estimates of the true and false positive rates, and their trade-off plot is known as a receiver operating characteristic (ROC) curve. Classification performance can be increased, however, if the information provided by multiple systems can be fused together to create a new, combined system. This research focuses on label-fusion as a common method to increase classification performance and quantifying the bias that occurs when misspecifying the partitioning of the underlying event set. This partitioning will be defined in terms of what be called within and across label fusion. When incorrect assumptions are made about the partitioning of the event set, bias will occur and both the ROC curve and its optimal parameters will be incorrectly quantified. In this work, we analyze the effects of individual classification system performance, correlation, and target environment on the magnitude of this performance bias. This work will then inspire the development of formulas to adjust optimal performance measures to appropriately reflect the fused system performance according to event set partitioning. As such, bias may be appropriately adjusted without redesigning the fused system, allowing greater use of currently fused systems across multiple platforms and environments.