ProtoViewer: Visual Interpretation and Diagnostics of Deep Neural Networks with Factorized Prototypes

摘要

In recent years deep neural networks (DNNs) are increasingly used in a variety of application domains for their state-of-the-art performance in many challenging machine learning tasks. However their lack of interpretability could cause trustability and fairness issues and also makes model diagnostics a difficult task. In this paper we present a novel visual analytics framework to interpret and diagnose DNNs. Our approach utilizes ProtoFac to factorize the latent representations in DNNs into weighted combinations of prototypes, which are exemplar cases (e.g., representative image patches) from the original data. The visual interface uses the factorized prototypes to summarize and explain the model behaviour as well as support comparisons across subsets of data such that the users can form a hypothesis about the model’s failure on certain subsets. The method is model-agnostic and provides global explanation of the model behaviour. Furthermore, the system selects prototypes and weights that faithfully represents the model under analysis by mimicking its latent representation and predictions. Example usage scenarios on two DNN architectures and two datasets illustrates the effectiveness and general applicability of the proposed approach.