Predicting Learners' Emotions in Mobile MOOC Learning via a Multimodal Intelligent Tutor

摘要

Massive Open Online Courses (MOOCs) are a promising approach for scalable knowledge dissemination. However, they also face major challenges such as low engagement, low retention rate, and lack of personalization. We propose AttentiveLearner~2, a multimodal intelligent tutor running on unmodified smartphones, to supplement today's clickstream-based learning analytics for MOOCs. AttentiveLearner~2 uses both the front and back cameras of a smartphone as two complementary and fine-grained feedback channels in real time: the back camera monitors learners' photoplethysmography (PPG) signals and the front camera tracks their facial expressions during MOOC learning. AttentiveLearner2 implicitly infers learners' affective and cognitive states during learning from their PPG signals and facial expressions. Through a 26-participant user study, we found that: (1) AttentiveLearner~2 can detect 6 emotions in mobile MOOC learning reliably with high accuracy (average accuracy = 84.4%); (2) the detected emotions can predict learning outcomes (best R~2 = 50.6%); and (3) it is feasible to track both PPG signals and facial expressions in real time in a scalable manner on today's unmodified smartphones.