The psychometric equivalency of scores from a Web-based questionnaire administered via cellphone versus desktop computer.

摘要

This study investigates the psychometric issues and viability of cellphone-based-testing, a novel test administration modality whereby test-takers use a cellphone to respond to items on a web-based assessment. The study explored mode-dependent differences in scores from a web-based version of the Self-Monitoring Scale (SMS) administered across two modalities: desktop computer and cellphone. The selection of the SMS was based on several pre-established criteria. The instrument was simple and brief. Its text-based items included true/false response categories. Its rights of use fell under public domain and it had been previously validated for online administration.; The study includes a comprehensive overview of recent literature related to the topic of psychometric equivalency and incorporates numerous methodological approaches to determine test score equivalence, including: comparisons of central tendency, dispersions, and rank order; the Kolmogorov-Smirnov test of equal score distributions; the Pitman procedure for detecting differences in reliability coefficients; a confirmatory factor analysis of the equality of factor structures using LISREL; and an analysis of differential item functioning based on item response theory using BILOG-MG.; The study employed a counterbalanced repeated measures design whereby 234 participants took an 18-item web-based version of the SMS using a desktop computer and/or a cellphone. The psychometric equivalency of scores from the two modes of administration was analyzed. All statistical comparisons provided overwhelming support for one general conclusion: There were no mode-dependent differences in scores on the web-based version of the SMS when administered by desktop computer versus cellphone.; The study also explored participants' attitudes toward using cellphones as a test-taking tool. The participants correctly anticipated that their scores would not be affected by using a cellphone, but they categorically rated the cellphone as less enjoyable, more difficult, and more cumbersome than a desktop computer. However, one cannot ignore the tendency of our modern society for being obsessed with information on demand. As cellphone technology continues to improve and the text-messaging generation begins to influence the field of educational and psychological measurement, cellphone-based-testing will likely become an accepted standard for both academic and clinical practice.