Spam Filter Optimality Based on Signal Detection Theory

摘要

Unsolicited bulk email, commonly known as spam, represents a significant problem on the Internet. The seriousness of the situation is reflected by the fact that approximately 97% of the total e-mail traffic currently (2009) is spam. To fight this problem, various anti-spam methods have been proposed and are implemented to filter out spam before it gets delivered to recipients, but none of these methods are entirely satisfactory. In this paper we analyze the properties of spam filters from the viewpoint of Signal Detection Theory (SDT). The Bayesian approach of Signal Detection Theory provides a basis for determining the optimality of spam filters, i.e. whether they provide positive utility to users. In the process of decision making by a spam filter various tradeoffs are considered as a function of the costs of incorrect decisions and the benefits of correct decisions.