Evaluation of Information Professionals Competency Face Validity Test Using Rasch Model

摘要

The development of Rasch Measurement Model in social science educational measurement has rapidly expanded to other areas of education including technical and engineering fields. Originally, there was substantial controversy between those who saw Rasch Model as a relevant method of measurement in technical fields and those who saw them as essentially different. This paper is an attempt of a paradigm shift in testing and validating a process towards bio-based Rasch Model. It is believed compatibility exist with the fundamental measurement currently used based on Kuhn's explanation on the role of measurement in physical science particularly in measuring competency which is categorised as latent trait. These cannot be gleaned from textbooks in computer engineering or statistics. Taking the paradigm shift, many technical faculties in Institutions of Higher Learning has embarked on the application of Rasch Model to measure the achievement of it's program Learning Outcomes (LO). Face validity tests were conducted subsequent to rigorous meta-analysis on the attributes identified from literature reviews. The major constraint in face validity test is the very small number of sample that is involved; hence reliability. Rasch Model tabulates these expert's opinion on a Person and Items Distribution Map (PIDM) which gives a summative overview on their Level of Agreement for the attributes duly identified. Comparative analysis against the traditional t-test to show the corelation between the experts and the attributes shows that Rasch measurement was found to give a better exploratory depth in understanding the experts level of agreement of an attribute. Despite the small sample size, the experts opinion were clearly defined as to their level of acceptance according to the respective dimension before an attribute shall be considered for the development of survey questionaires as the research instrument; hence construct validity. This is of utmost importance as a bad construct is detrimental to a research findings.