METHODS AND SYSTEMS FOR RUNNING DYNAMIC RECURRENT NEURAL NETWORKS IN HARDWARE

摘要

A method of implementing in hardware a recurrent neural network (RNN) in which each step operates on a different input in a sequence of inputs, such as a time series. A representation of the RNN is transformed into a derivative neural network which operates over a defined plurality of inputs from the sequence of inputs and is equivalent to a plurality of steps of the RNN, e.g. by unrolling the RNN over the plurality of steps. The derivative neural network is iteratively applied to the sequence of inputs by implementing a sequence of instances of the derivative neural network in hardware, and providing the output of each instance of the derivative neural network as the input to the subsequent derivative neural network, so as to operate the RNN over a sequence of inputs longer than the defined plurality of inputs. The RNN may comprise cells. Transforming the RNN may include each cell identifying non-causal operations which are performed independently of the cell’s input, and the derivative neural network grouping together non-causal operations at the cell for parallel processing, e.g. as a single convolution operation. The hardware may include an accelerator having processors adapted to perform convolution operations.