Weapons of Math Destruction

摘要

We live in the era of "big data" which, ultimately, originated in the burgeoning human population. In a sense the modern computer began with Hollerith's "tabulating machine" used to process and tabulate the 1890 census of the US population.1,2 Large-scale "number crunching" is used nowadays by governments to monitor the citizens they govern, by scientists to systematize their observations, by engineers to simulate systems too complex for analytical solutions, and by private companies to maximize income. This development has been enabled not only by the enormous increase in computing power and the enormous decrease in the cost of data storage ("memory"), but also by the development of sophisticated hardware with which the data can be gathered. For example, the People's Republic of China's "social credit" system relies, inter alia, on ubiquitous urban surveillance cameras. Whole-genome sequencing relies on the fragmentation of the genome, sequence determination of the fragments, and a unique, most probable, reconstruction using pattern-matching algorithms. Finite element analysis is ubiquitous in the engineering world as a way of testing new structures and devices and optimizing their performances. Cellphones, the Internet and electronic payment systems enable enormous volumes of purchasing transactions to be recorded and analysed; in the relentless search for patterns, algorithms may evolve whose inner workings are opaque to the human observer.3 Although Western observers see China's social credit project as an enormous erosion of personal freedom by the State, private companies in the West, albeit in the more fragmented fashion, are doing much the same thing: applications for insurance policies, personal loans and jobs will all nowadays be initially scrutinized by algorithms to screen out unsuitable applicants.