Automated assessment of Parkinsonian finger-tapping tests through a vision-based fine-grained classification model

摘要

Movement disorder of Parkinson & rsquo;s disease (PD) is usually quantified by the Movement Disorders Society sponsored Revision of the Unified Parkinson & rsquo;s Disease Rating Scale (MDS-UPDRS) to evaluate its severity. However, the lack of well-trained experts and subjective inter-rater variability often limit an effective and objective assessment in clinical practice. Hence, developing an automated assessment method for movement disorders in PD is crucial. Here, we present a novel vision-based fine-grained action recognition model to cope with one of the most critical and challenging tasks in clinical scales: the finger-tapping test. Specifically, we establish a three-stream fine-grained classification network with a Markov chain fusion model to aggregate multi-stream information of the skeleton sequence from finger-tapping test videos. Then, we develop a spatial & ndash;temporal attention mechanism to capture rich spatial and temporal long-range dependencies from skeleton data and introduce a symmetric bilinear pooling layer to enrich the local feature representation of each stream & rsquo;s output. Besides, a mini-batch-based balanced algorithm is designed to ensure that the samples in each mini-batch are inter-class balanced, thus mitigating the effect of imbalanced data on neural networks. Finally, our three-stream fine-grained classification network achieved an accuracy of 72.4% and an acceptable accuracy of 98.3% on 157 patients and 744 videos. Extensive experiments further confirm our approach & rsquo;s effectiveness and reliability. This method does not require any wearable device and has excellent potential for remote monitoring of PD patients in the future.(c) 2021 Elsevier B.V. All rights reserved.