'Cargo cult' science in traditional organization information systems survey research: A case for using nontraditional methods of data collection, including Mechanical Turk and online panels

摘要

This paper evaluates information technology (IT) employees on service christened as internal IT service perceptions. The article uses MTurk as an online source of data to access many IT employees. Three contentions are set forth that lie beyond contradistinction between shared and individual cognition. First is that MTurk samples are spread geographically and on backgrounds compared to a sample within an organization. These differences make the data biased and not applicable in the context of the organization. Third due to broad variability, the outcomes can be generalized, but are not specific. These criticisms are not standing since many traditional researchers reject new approaches and support traditional paper-based data collection simply because they are used to it. It is worth mentioning that the drawbacks highlighted by traditionalists are valid for paper-based data collection also. Just because one practice is accepted in the past doesn't mean improvements cannot be considered. Researchers should adopt more scientific methods in data collection. Many arguments are similar to ones faced by online surveys when first introduced. The article refutes criticisms against MTurk and provides scientific guidelines for using it. (52 refs.)