ON PERFORMANCE EVALUATION OF PROGNOSTICS ALGORITHMS

摘要

Prognostics and Health Management, as an emerging engineering discipline, has been facing difficulties in algorithm performance evaluation, validation and benchmarking. Over years, various metrics have been developed to assess algorithm performance from different perspectives. Lack of methodology in selecting metrics from the large metric pool has made it hard for a business case of prognostics to be translated into measurable technical metrics to guide algorithm development. The situation, however, may be alleviated if the many performance metrics are properly prioritized from the end-user point of view. In this paper, we propose to adopt the simple but widely accepted performance metrics from classification discipline. The basic measures such as False Positive, False Negative, etc. and the performance metrics derived wherefrom are redefined in prognostics context. And then a new concept called algorithm Performance Profile, which characterizes the performance of an algorithm by the accuracy score at each estimated RUL (Remaining Useful Life), are proposed. The Performance Profile is obtained during design phase, and will become a priori at run time of the algorithm. It can be interpreted as the algorithm's trustworthiness score for people to make maintenance decisions, or utilized by the maintenance scheduling program to assess risks in response to a given RUL prediction.