DISCOVERY OF UNUSUAL, UNEXPECTED, OR ANOMALOUS INFORMATION AND TRENDS IN HIGH THROUGHPUT DATA STREAMS AND DATABASES USING PROBABILITSTIC SURPRISAL CONTEXT FILTERS

摘要

A method, system, and computer program product for detecting anomalous events from a data input comprising a plurality of events. The method comprising the steps of: selecting at least one filter selecting for context data determined to be probabilistically present within a specified degree of certainty in the data input; comparing the data input to the selected at least one filter; discarding the events from the data input that are the same as the context data for which the at least one filter selects; and storing in a repository the events remaining in the data input as anomalous events.