Using postgraduate students' evaluations ofresearch experience to benchmark departmentsand faculties: Issues and challenges

摘要

Background. The introduction of the Australian Research Training Scheme has beena strong reason for assuring the quality of the research higher degree (RHD)experience; if students experience poor supervision, an unsupportive climate, andinadequate infrastructure, prior research suggests RHD students will be less likely tocomplete their degree, with negative consequences for the student, the university, andsociety at large. Aims. The present study examines the psychometric properties of a surveyinstrument, the Student Research Experience Questionnaire (SREQ), for measuring theRHD experience of currently enrolled students. The core scales of the SREQ focus onstudent experiences of Supervision; Infrastructure; Intellectual and Social Climate; andGeneric Skills Development. Sample. Participants were 2,213 postgraduate research students of a large,research-intensive Australian university. Results. Preliminary factor analyses conducted at the student level supported thea priori four factors that the SREQ was designed to measure. However, multi-levelanalyses indicated that there was almost no differentiation between faculties ordepartments nested with faculties, suggesting that the SREQ responses are notappropriate for benchmarking faculties or departments. Conclusions. Consistent with earlier research based on comparisons acrossuniversities, the SREQ is shown to be almost completely unreliable in terms ofbenchmarking faculties or departments within a university.