SYSTEMS, APPARATUS, METHODS, AND ARCHITECTURE FOR PRECISION HETEROGENEITY IN ACCELERATING NEURAL NETWORKS FOR INFERENCE AND TRAINING

摘要

For one embodiment, a hardware accelerator with a heterogenous architecture for training quantized neural networks is described. In one example, a hardware accelerator for training quantized data, comprises memory to store data, a plurality of compute units to perform computations of a data type for an inference phase of training quantized data of a neural network, and plurality of heterogenous precision compute units to perform computations of mixed precision data types for a backward propagation phase of training quantized data of the neural network.