Exploring RRAM Variability as Synapses on Inception Simulation Framework to Characterize the Prediction Accuracy and Power Estimation per Bit for Convolution Neural Network

摘要

Resistive random access memory (RRAM) is one of the most preferred candidates for implementation of hardware-based neural networks for future edge computing applications given its analog conductance behavior (depending on the material stack), ease of integration with the Si CMOS process, relatively lower power consumption (compared to traditional bulk phase change RAM devices) and high integration density. The matrix convolution operation is an inevitable process in a convolutional neural network (CNN) due to its robust approximation to learn and classify the non-linear relationship between the input and output data sets. Today, most of the popular CNNs are stacked with more and more convolution layers (referred to as deep learning) to improve performance, but in many instances, this approach tends to over fit the data, resulting in prediction accuracy loss. The Inception network was an essential milestone in the development of CNN classifiers. The Inception layer is constructed with parallel pipelines of convolution operators, which resulted in improved performance. While several studies have focused on the quantification of the impact of RRAM degradation on the prediction accuracy for pattern classification / image recognition in a simple arbitrary neural network with one or two hidden layers, the impact of these hardware variations on a full-fledged convolutional neural network (CNN) that is commercially used is not well explored. In this study, we extract the GPU trained weights of the CNN platform for visual recognition and replace the GPU weights with the RRAM resistance data in the floating-point format for the Inception network layers alone to quantify the impact of “partial” hardware-based CNN prediction accuracy.