Algorithmically outsourcing the detection of statistical errors and other problems

摘要

Software to check texts for spelling errors is commonplace, but catching errors of a more technical nature, such as incorrect P‐value calculations, is still a manual endeavor. Nonetheless, text‐mining technology to catch a growing number of error types within scientific manuscripts has been developed by studies interested in broad, literature‐wide surveys. The same algorithms that are now used to retrospectively identify potential errors in published papers can also be used pre‐emptively to identify errors before publication. So far, these algorithms have focused on finding errors of commission, such as incorrect calculations, but could also find errors of omission, such as leaving out details needed to reproduce the results. This could offer many advantages for those aspects of peer review that are amenable to double‐checking by an algorithm: consistency, uniformity, speed, cost efficiency, and reducing the growing burden on peer reviewers.