A Remote Free-Head Pupillometry Based on Deep Learning and Binocular System

摘要

Objective: Pupillometer plays a key role in a variety of research areas, including disease diagnosis, human-machine interaction, and education. Here, we set out to leverage the deep learning theory to develop a remote binocular vision system for pupil diameter estimation. Approach: the system consists of three parts: eye detection, eye tracking, and pupil diameter estimation. We first train a convolutional neural network based on YOLO V2 to perform eye detection, leading to high accuracy and robustness under ambient light interference. By exploring the similarity of binocular camera images, we then propose a master-slave structure for eye tracking, surpassing the traditional parallel structure in tracking speed while keeping considerable accuracy. Furthermore, we develop a pupil diameter estimation algorithm based on binocular vision, avoiding the personal calibration procedure and reducing the measurement distortion error. Main results: Experimental results on real datasets reveal that our system exhibits the state-of-the-art performance with high eye detection accuracy (90.6%), fast eye tracking speed (<11 ms per frame), low pupil diameter estimation error [(0.022 +/- 0.017) mm mean absolute error, and (0.6 +/- 0.7)% percentage of the mean absolute errorl and excellent flexibility. Significance: in contrast with previous pupillometers, which lead to pupil diameter measurement distortion error through a 2-D projection image on a single camera, our system measures pupil diameter in 3-D space without distortion influence, thus improving its robustness to head angle variation and making it more practical for real applications.