首页>
外国专利>
Initializing and learning rate adjustment for rectifier linear unit based artificial neural networks
Initializing and learning rate adjustment for rectifier linear unit based artificial neural networks
展开▼
机译:基于整流器线性单元的人工神经网络的初始化和学习率调整
展开▼
页面导航
摘要
著录项
相似文献
摘要
A data processing technique uses an Artificial Neural Network (ANN) with Rectifier Linear Units (ReLU) to yield improve accuracy in a runtime task, for example, in processing audio-based data acquired by a speech-enabled device. The technique includes a first aspect that relates to initialization of the ANN weights to initially yield a high fraction of positive outputs from the ReLU. These weights are then modified using an iterative procedure in which the weights are incrementally updated. A second aspect relates to controlling the size of the incremental updates (a “learning rate”) during the iterations of training according to a variance of the weights at each layer.
展开▼