An Evaluation Framework for Web-based Aptitude Tests

摘要

Aptitude tests analyse the aptitude of persons for studying at a specific school or university as well as for working within a specific company. Due to the recent technology advances, web-based solutions are increasingly used for the implementation of aptitude tests. These web-based aptitude tests can be utilised for rather standardized test methods, testing a large amount of users. Based on the fact that web-based aptitude tests are getting more and more common, a high user acceptance is important, especially since test results tend to be taken more seriously. Furthermore, the design of the test should be helpful and support the use of the test. In this context, the target of our research is to provide a framework for the evaluation of the user acceptance for web-based aptitude tests. The research method is based on an exemplary web-based aptitude test and includes the following steps: Firstly, we used the Dynamic Acceptance Model for the Re-evaluation of Technologies (DART) as a basis for the adoption of web-based aptitude tests. DART is specially designed for analysing and evaluating user acceptance of innovative technologies. Based on a literature review and expert interviews, we identified the most important acceptance indicators. Our results showed that the use of the DART-approach helps to define a balanced set of measurable indicators for the evaluation of the user acceptance. In a next step, we evaluated the defined acceptance indicators in a survey with test persons who carried out one selected web-based aptitude test. Finally, we analysed the reliability and validity of the developed evaluation framework. The analysis showed that a revision of some of the indicators and items would be useful. Based on our results, we described lessons learned and the ongoing process to measure the acceptance of web-based aptitude tests.