Using scantron versus an audience response system for survey research: Does methodology matter when measuring computer-mediated communication competence?

摘要

The purpose of this study was to investigate whether results of the computer-mediated communication (CMC) competency scale differ depending on the survey instrument used. Data were collected from 173 undergraduate students (53% female; average age 20). Of these, 96 subjects completed the questionnaire using scantron sheets, while 77 subjects used an audience response system called Classroom Performance System (CPS). All subjects completed a second questionnaire using paper-and-pencil to evaluate their experience with either the scantron sheets or the CPS. t-Test results show that using the CPS is more fun for participants (p < .000); no more or less difficult (p = .806); and does not make a difference with regard to experienced time pressure (p = .425). Subjects have higher doubts about the validity of their recorded answers (p = .007) when using the CPS. Regression analysis shows that in two out of 12 constructs of the CMC competency scale, scoring was influenced by using a technological data collection method. The two constructs were "message factors" (p = .002) and "medium factors" (p = .002). Overall, results support the use of the CPS as an alternative to scantron sheets for measuring CMC competence.