Eyes speak out Mind: Deep models for Gaze-based Analysis of Bilingual And Monolingual Reading

摘要

This paper presents an approach that attempts to explore effects of bilingualism in reading by analysis of eye-gaze data. As readers are more comfortable in reading first language (L1) compared to second language (L2), their eyes move differently in these two cases. Being motivated by this hypothesis, a deep learning model is developed to predict whether a reader is reading first or second language by analyzing the reading behavior given by the eye-tracking data. Exposure to different languages simultaneously may influence the proficiency in native language reading and this matter is also investigated by using deep learning model which can differentiate monolingual readers from bilingual readers analyzing their eye movement during reading in their respective native language. Experiments are conducted on GECO Corpus [1] which provides eye-tracking data of 19 readers reading a novel in L1 and L2 along with eye-tracking data of several monolingual readers reading same text in their native language. Experimental results show that the proposed models are quite efficient in capturing the differences in eye gaze pattern for reading L1 and L2 for a reader as well as in identifying monolingual and bilingual readers by processing eye-gaze data.