Montage based 3D Medical Image Retrieval from Traumatic Brain Injury Cohort using Deep Convolutional Neural Network

摘要

Brain imaging analysis on clinically acquired computed tomography (CT) is essential for the diagnosis, risk prediction ofprogression, and treatment of the structural phenotypes of traumatic brain injury (TBI). However, in real clinical imagingscenarios, entire body CT images (e.g., neck, abdomen, chest, pelvis) are typically captured along with whole brain CTscans. For instance, in a typical sample of clinical TBI imaging cohort, only ～15% of CT scans actually contain wholebrain CT images suitable for volumetric brain analyses; the remaining are partial brain or non-brain images. Therefore, amanual image retrieval process is typically required to isolate the whole brain CT scans from the entire cohort. However,the manual image retrieval is time and resource consuming and even more difficult for the larger cohorts. To alleviate themanual efforts, in this paper we propose an automated 3D medical image retrieval pipeline, called deep montage-basedimage retrieval (dMIR), which performs classification on 2D montage images via a deep convolutional neural network.The novelty of the proposed method for image processing is to characterize the medical image retrieval task based on themontage images. In a cohort of 2000 clinically acquired TBI scans, 794 scans were used as training data, 206 scans wereused as validation data, and the remaining 1000 scans were used as testing data. The proposed achieved accuracy=1.0,recall=1.0, precision=1.0, f1=1.0 for validation data, while achieved accuracy=0.988, recall=0.962, precision=0.962,f1=0.962 for testing data. Thus, the proposed dMIR is able to perform accurate CT whole brain image retrieval from largescaleclinical cohorts.