Robust regression methods for massively decayed intelligence data.

摘要

Homeland Security, sponsored by governmental initiatives, has become a vibrant academic research field. However, most efforts were placed with the recognition of threats (e.g. theory) and response options. Less effort was placed in the analysis of the collected data through statistical modeling. In a field that collects more than 20 terabyte of information per minute though diverse overt and covert means and indexes it for future research, understanding how different statistical models behave when it comes to massively decayed data is of vital importance.;Using Monte Carlo methods, three regression techniques (ordinary least squares, least-trimmed, and maximum likelihood) were tested against different data decay models presumed to be found in homeland security research studies in order to test whether these techniques will preserve the Type I error rate in the t-test of standardized beta.;The results of these Monte Carlo simulations (sample size n=30,90,120,240,480 and 100,000 iterations) showed that the least trimmed squares method should be avoided under any circumstance due to the lack of a defined standard error, while the maximum likelihood technique should be avoided with smaller sample sizes due to the inflated Type I errors. Interestingly, although it is known that the ordinary least squares regression can be impacted by non-normality and other assumption violations, it is remarkable robust to normally distributed data that is subject to massive decay.;Keywords: Homeland Security, Analysis, Data Decay, Monte Carlo, Regression.