A Novel Method of AOT Evaluation for Change-Point Detection on Synthetic Time-series

摘要

On the basis of the empirical ROC (Receiver Operating Characteristic) analysis, we propose a new AOT (Area out of Triangle) method for evaluating a change-point detection model. In this AOT method, we borrow the conceptions of ROC analysis and take the area out of a triangle formed by TPR (True Positive Rate) and FPR (False Positive rate), as a measurement of the search space during change-point detection on multiple time-series samples. Based on synthetic time-series test groups, the proposed AOT is applied to evaluate the different KS, T and SSA change-point detection models. In terms of AOT and other computation time, hit rate and accuracy measurements, the simulated studies indicate that our AOT can efficiently evaluate the search space for different change-point detection models on the left, middle and right boundaries in test sample groups respectively. Especially, the results suggest that the KS needs much smaller values of search space, computation time, and has much better search capability and search efficiency than T and SSA.