(616)THE IMPACT ON THE QUALITY AND SATISFACTION OF THE TESTS OF AN UNCUED QUESTION FORMAT VERSUS A SHORT ANSWER QUESTION FORMAT

摘要

Objectives: To assess the reliability and quality of an uncued question format (UQF), when comparedto a short answer question format (SAF)in the course of Biopathology II at the Faculty of Medicine ofPorto University and compare the satisfaction of students with the introduction of a new glossarybased on an uncued question format.Methods: Two hundred and eighteen students (218)were tested using SAF and 283 students wereassessed using UQF. We resorted to both the Classical Test Theory (CTT)and the Item ResponseTheory (IRT)to compare both types of questions.To assess the level of satisfaction with the new glossary, we analyzed the answers of 227 studentsthat fulfilled a questionnaire that evaluates 3 dimensions (Satisfaction with new glossary (G1);;Satisfaction with new answer sheet (G2);;Model preference (G3))ranged from 1 to 5 and BiopathologyI classification (G4)). The Spearman's Correlation test was applied between dimensions.Results: Trough the CTT, we determined that the differences between the Difficulty Index on the firsttest (0.674 +/- 0.291)and on the second test (0.638 +/- 0.303)were not significant (p=0.367). Inaddition, the differences found on the Discrimination Index of the first and second test (0.318+/-0.116and 0.300+/-0.122, respectively)were not statistically different (p=0.267). Using the IRT, the DifficultyIndex (SAF: 0.698 +/- 0.312 and UQF: 0.655 +/- 0.325 after)and the Discrimination Index (SAF: 0.685+/- 0.163 and UQF: 0.639 +/- 0.203)were not statistically different between both tests (p=0.318 andp=0.0698, respectively). Results indicate that both approaches were reliable, with Cronbach's Alpha of0.9298 on the SAF and 0.9760 on the UQF.The students disagreed with the change in the model of evaluation (Mean=3.84). The analysis of thesatisfaction questionnaires showed that students who preferred the UQF also preferred the newanswer sheet and/or the new glossary. In general, students were not satisfied with the answer sheet.Similarly, students who were not satisfied with the UQF (answer sheet and/or glossary)preferred theprevious model of evaluation (rho GI/GIII=-0.664 and rho GII/GIII=-0.542). We found no correlationbetween the student's classification and their preference with the new model of evaluation (p=0.428)Conclusions: The change of format did not have an effect in the quality of the test, however thestudents were not very satisfied with the new format independently of the student's classification.Even though this format did not decrease the quality parameters it was a more efficient tool in terms oftime and resources saved by the correctors and faculty, in general.