The importance of prewhitening in change point analysis under persistence

摘要

The presence of serial correlation in hydrometeorological time series often makes the detection of deterministic gradual or abrupt changes with tests such as Mann-Kendall (MK) and Pettitt problematic. In this study we investigate the adverse impact of serial correlation on change point analyses performed by the Pettitt test. Building on methods developed for the MK test, different pre-whitening procedures devised to remove the serial correlation are examined, and the effects of the sample size and strength of serial dependence on their performance are tested by Monte Carlo experiments involving the first-order autoregressive [AR(1)] process, fractional Gaussian noise (fGn), and fractionally integrated autoregressive [ARFI-MA(1,d,0)] model. Results show that (1) the serial correlation affects the Pettitt test more than tests for slowly varying monotonic trends such as the MK test both for short-range and long-range persistence; (2) the most efficient prewhitening procedure based on AR(1) involves the simultaneous estimation of step change and lag-1 auto-correlation rho, and bias correction of rho estimates; (3) as expected, the effectiveness of the prewhitening procedure strongly depends upon the model selected to remove the serial correlation; (4) prewhitening procedures allow for a better control of the type I error resulting in rejection rates reasonably close to the nominal values. As ancillary results, (5) we show the ineffectiveness of the original formulation of the so-called trend-free prewhitening (TFPW) method and provide analytical results supporting a corrected version called TFPWcu; and (6) we propose an improved twostage bias correction of rho estimates for AR(1) signals.