首页> 外文期刊>Neural computation >Neural Associative Memory with Optimal Bayesian Learning
【24h】

Neural Associative Memory with Optimal Bayesian Learning

机译:最佳贝叶斯学习的神经联想记忆

获取原文

摘要

Neural associative memories are perceptron-like single-layer networks with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. Previous work optimized the memory capacity for various models of synaptic learning: linear Hopfield-type rules, the Willshaw model employing binary synapses, or the BCPNN rule of Lansner and Ekeberg, for example. Here I show that all of these previous models are limit cases of a general optimal model where synaptic learning is determined by probabilistic Bayesian considerations. Asymptotically, for large networks and very sparse neuron activity, the Bayesian model becomes identical to an inhibitory implementation of the Willshaw and BCPNN-type models. For less sparse patterns, the Bayesian model becomes identical to Hopfield-type networks employing the co-variance rule. For intermediate sparseness or finite networks, the optimal Bayesian learning rule differs from the previous models and can significantly improve memory performance. I also provide a unified analytical framework to determine memory capacity at a given output noise level that links approaches based on mutual information, Hamming distance, and signal-to-noise ratio.
机译:神经联想记忆是类似感知器的单层网络,具有快速的突触学习功能,通常存储成对的神经活动模式之间的离散联想。先前的工作为各种突触学习模型优化了存储容量:例如,线性Hopfield型规则,采用二元突触的Willshaw模型或Lansner和Ekeberg的BCPNN规则。在这里,我证明所有这些先前模型都是一般最佳模型的极限情况,在这种情况下,突触学习是由概率贝叶斯考虑因素决定的。渐近地,对于大型网络和非常稀疏的神经元活动,贝叶斯模型变得与Willshaw和BCPNN型模型的抑制实现相同。对于较少的稀疏模式,贝叶斯模型变得与采用协方差规则的Hopfield型网络相同。对于中等稀疏性或有限网络,最佳贝叶斯学习规则与以前的模型不同,可以显着提高内存性能。我还提供了一个统一的分析框架来确定给定输出噪声水平下的存储容量,该存储空间基于互信息,汉明距离和信噪比将方法链接在一起。

著录项

  • 来源
    《Neural computation》 |2011年第6期|p.1393-1451|共59页
  • 作者

    Andreas Knoblauch;

  • 作者单位

    Honda Research Institute Europe GmbH, D-63073 Offenbach, Germany;

  • 收录信息 美国《科学引文索引》(SCI);美国《化学文摘》(CA);
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号