Artificial Intelligence Methods to Forecast Engineering Students' Retention based on Cognitive and Non-cognitive Factors

摘要

Engineering students' affective self-beliefs can be influential factors directly or indirectly affecting their academic success and career decision. This paper examines whether students' non-cognitive factors can be used, alone or in combination with cognitive factors, in artificial neural network (ANN) models to predict engineering student's future retention. Four ANN based retention prediction models using different combinations of non-cognitive and cognitive factors are presented. The independent variables includes survey items from nine non-cognitive constructs (leadership, deep learning, surface learning, teamwork, self-efficacy, motivation, meta-cognition, expectancy-value, and major decision) and eleven cognitive items representing student's high school academic performance. The dependent variable (i.e., the output from these models) is the student's retention status after one year. Data from more than 4900 first-year engineering students from three freshman cohorts (2004, 2005, 2006) in a large Midwestern university were collected and utilized in training and testing these ANN prediction models. Among the four ANN models developed, the model combining 11 cognitive items and 60 selected non-cognitive items has the highest overall prediction accuracy at 71.3%, probability of detection (POD) for retained students at 78.7% and POD for not retained student at 40.5%. Removing the 11 cognitive items from this model, the overall prediction accuracy would drop slightly to 70.5%. Results from training and testing the same model using student data from different cohorts indicate the ANN model's predictive performance is generally stable across different cohort years. Also, a model trained with earlier year (2004) freshman cohort's data has maintained its predictive power very well when tested with student data from later (2005 and 2006) cohorts.