A Multimodal Learning Framework to Study Varying Information Complexity in Structural and Functional Sub-Domains in Schizophrenia

摘要

Approaches involving the use of learning architectures on multimodal neuroimaging data tend to assume uniformity in the way information is stored in various sub-domains of the brain, thus not catering to the differences across functional and structural sub-domains. We introduce a learning framework to effectively incorporate multimodal features using structural and functional MRI data from a dataset of schizophrenia patients and controls, accounting for and exploiting the heterogeneity in the sub-domains of the brain. We analyze these sub-domains in terms of their functional interactions (i.e. within and between network connectivity) and structural properties (gray matter volume). By using Bayesian optimization on a search space of flexible multimodal architectures with multiple branches, we demonstrate that the discriminatory information from structural and functional sub-domains can be better recovered if the complexity of subspace structure in the model can be tuned to reflect the extent of non-linearity with which each sub-domain encodes the information. Our repeated cross-validated results from a schizophrenia classification problem show that for better classification and interpretation, sub-domains known for their role or disruption in Schizophrenia require more sophisticated subspace structure in the model compared to others. Our work emphasizes on the requirement to create multimodal frameworks that can adapt based on differences in the way various sub-domains of the brain encode discriminatory information. This is important to not only have better-performing prediction models but also to reveal sub-domains associated with the outcome at hand.