A Multimodal Diagnosis Predictive Model of Alzheimer’s Disease with Few-shot Learning

摘要

Alzheimer's disease imposes a significant quality of life and socioeconomic burden on patients, while mild cognitive impairment is considered as the first stage of Alzheimer's disease. Therefore, being able to detect and treat the disease while it is in mild cognitive impairment is essential for effective prevention of Alzheimer's disease. Many scholars have proposed a variety of different AI diagnostic models for Alzheimer's disease. These models are mainly trained based on MRI image data. However, in addition to MRI images, clinical diagnosis data of Alzheimer's disease also includes data of multiple modalities such as MRI text reports and test results of mental and psychological scales. Moreover, in the actual clinical data, there are fewer samples of Alzheimer's disease data and lack of sufficient training samples. This paper proposes a classification and diagnosis method for Alzheimer's patients based on multi-modal feature fusion and small sample learning. Specifically, we use structured MRI image reports and corresponding MRI images as multi-modal input, and then the compressed interactive network is used to explicitly fuse the extracted features at the vector level; finally, the KNN attention pooling layer and the convolutional network are used to construct a small sample learning network to classify the patient diagnosis data. The experimental results show that the accuracy and Fl-score value based on multi-modality are improved by more than 10% compared with single-modality. Using the small sample learning method under the same multimodal data, the accuracy rate increased by 8.2%, and the Fl-score value increased by 8.4%.