USING MACHINE LEARNING TO PREDICT DISEASE IN TRANSITION PERIODDAIRY CATTLE - THE CHALLENGES OF AN IMBALANCED DATASET AND THEIMPORTANCE OF METRICS

摘要

In our study, while attempting to fit predictive models for transition cow disease, it became evident that the metrics chosen for reporting can lead to misleading interpretation of the model's performance. When building classification algorithms, in which the goal is to correctly assign new data to predetermined classes, it is common that only accuracy, sensitivity and specificity are reported. For imbalanced datasets however, where the number of data points for each class differs substantially, theomission of metrics such as kappa or balanced accuracy can lead to inaccurate conclusions, especially since the overall accuracy is commonly very high. In this research, we illustrate the importance of reporting all relevant metrics associated with a model's performance in order to reflect the model's genuine predictive capability with new data.