Imitating Pathologist Based Assessment With Interpretable and ContextBased Neural Network Modeling of Histology Images

摘要

Convolutional neural networks (CNNs) have gained steady popularity as a tool to perform automatic classification of whole slide histology images. While CNNs have proven to be powerful classifiers in this context, they fail to explain this classification, as the network engineered features used for modeling and classification are ONLY interpretable by the CNNs themselves. This work aims at enhancing a traditional neural network model to perform histology image modeling, patient classification, and interpretation of the distinctive features identified by the network within the histology whole slide images (WSIs). We synthesize a workflow which (a) intelligently samples the training data by automatically selecting only image areas that display visible disease-relevant tissue state and (b) isolates regions most pertinent to the trained CNN prediction and translates them to observable and qualitative features such as color, intensity, cell and tissue morphology and texture. We use the Cancer Genome Atlas’s Breast Invasive Carcinoma (TCGA-BRCA) histology dataset to build a model predicting patient attributes (disease stage and node status) and the tumor proliferation challenge (TUPAC 2016) breast cancer histology image repository to help identify disease-relevant tissue state (mitotic activity). Wefind that our enhanced CNN based workflow both increased patient attributepredictive accuracy (~2% increase for disease stage and ~10% increase for nodestatus) and experimentally proved that a data-driven CNN histology modelpredicting breast invasive carcinoma stages is highly sensitive to features suchas color, cell size, and shape, granularity, and uniformity. This worksummarizes the need for understanding the widely trusted models built using deeplearning and adds a layer of biological context to a technique that functionedas a classification only approach till now.