Transient Pattern Detection from Streaming Nature Data

摘要

A novel challenging for scientific discovery is finding an un-expected phenomenon from nature data. For such data-intensive discovery, we study online algorithms for detecting transient patterns in streaming nature data. There are two issues to be addressed. First, due to the limitation of memory resources, we need to process every incoming data by on-the-fly manner. Secondly, we need to avoid the false alert as far as possible, which adversely impacts on decisions. For tackling the first issue, we propose a fast detection algorithm based on the Chebyshev inequality. The Chebyshev inequality is a classic but general tail- bound inequality satisfied over any observed data. Together with adaptive windowing technique, the proposed algorithm can quickly decide if the prior data is in a normal or transient zone. We then introduce a two-phase alert mechanism into the baseline, which is effective for reducing the false alert rate. Consequently, we demonstrate the performance of our proposed method obtained by using both the artificial data and real data.