Quantitative explorations of graduate learners' monitoring proficiencies and task understandings in the context of ill-structured writing assignments : from learner to work task as unit of analysis

摘要

Research has debated the degree of domain generality of monitoring skills through the theoretical lens of self-regulated learning, largely in the context of studies involving college/undergraduate-level objective, multiple-choice tests. The present quantitative study sheds some much-needed light on the nature of monitoring skills in 39 adult learners tackling ill-structured writing tasks for a graduate-level e-learning theory course in the domain of educational technology. Performance prediction and confidence in predictions were collected through a theoretically-grounded self-assessment tool termed TAPE (Task Analyzer and Performance Evaluator). Monitoring proficiencies were calculated using the instructor's assessment of performance and the TAPE-related measures. Using "learner" as unit of analysis, repeated measures procedures reveal improvements in the instructor's assessment of performance but not in any monitoring proficiencies. While the task-generality of the monitoring skills of discrimination and bias is confirmed through correlational analyses, facets of their specificities stand out due to the absence of intra-monitoring measure correlations. Subsequently, using the 247 instances of the writing task as unit of analysis, parametric multiple regression procedures demonstrate that 39% of variance in individual essay performance is predicted by combined variances in absolute prediction accuracy, discrimination, performance prediction and self-assessment scores. In addition, non-parametric ordinal and multinomial regression procedures reveal that individual essay performance can be predicted from the monitoring measures of bias, prediction confidence and absolute prediction accuracy, as well as from the self-assessment scores. The dual levels of analyses allow not only the quantitative description of learners' content-specific calibration of performance on a writing task, but also contextualized, essay-specific insight into how individual performance on an instance of the writing task is influenced by measures of monitoring and task understanding. Results are interpreted in light of the novel procedures undertaken in calculating monitoring measures like bias using the theoretical notion of performance prediction capability. Findings are also discussed with respect to the "work task as unit of analysis" approach which enables not only the generalization to the tasks completed for the specific course described in this study, but also the interchangeability of the tasks when treating variables such as time, class session, individual student and gender as fixed effects in the various regression approaches adopted for analyses