Unpaired whole-body MR to CT synthesis with correlation coefficient constrained adversarial learning

摘要

MR to CT image synthesis plays an important role in medical image analysis, and its applications included, but notlimited to PET-MR attenuation correction and MR only radiation therapy planning. Recently, deep learning-based imagesynthesis techniques have achieved much success. However, most of the current methods require large scales of paireddata from two different modalities, which greatly limits their usage as in some situation paired data is infeasible toobtain. Some efforts have been proposed to relax this constraint such as cycle-consistent adversarial networks (Cycle-GAN). However, the cycle consistency loss is an indirect structural similarity constraint of input and synthesized images,and it can lead to inferior synthesized results. To overcome this challenge, a novel correlation coefficient loss isproposed to directly enforce the structural similarity between MR and synthesized CT image, which can not onlyimprove the representation capability of the network but also guarantee the structure consistency between MR andsynthesized CT images. In addition, to overcome the problem of big variance in whole-body mapping, we use the multiviewadversarial learning scheme to combine the complementary information along different directions to provide morerobust synthesized results. Experimental results demonstrate that our method can achieve better MR to CT synthesisresults both qualitatively and quantitatively with unpaired MR and CT images compared with state-of-the-art methods.