首页> 外文期刊>Neural computation >Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units
【24h】

Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units

机译:离散单元的窄置信网络的通用逼近深度和误差

获取原文
获取原文并翻译 | 示例

摘要

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever & Hinton, 2008; Le Roux & Bengio, 2008, 2010; Montúfar & Ay, 2011) to units with arbitrary finite state spaces and the vanishing approximation error to an arbitrary approximation error tolerance. For example, we show that a q-ary deep belief network with src="/ielx7/6720226/6855468/6855489/abstract_images/NECO_a_00601inline1.gif" /> layers of width src="/ielx7/6720226/6855468/6855489/abstract_images/NECO_a_00601inline2.gif" /> for some src="/ielx7/6720226/6855468/6855489/abstract_images/NECO_a_00601inline3.gif" /> can approximate any probability distribution on src="/ielx7/6720226/6855468/6855489/abstract_images/NECO_a_00601inline4.gif" /> without exceeding a Kullback-Leibler divergence of src="/ielx7/6720226/6855468/6855489/abstract_images/NECO_a_00601inline5.gif" />. Our analysis covers discrete restricted Boltzmann machines and naive Bayes models as special cases.
机译:我们对最近的理论工作进行了概括,对狭窄深度信念网络的最小层数进行了研究,这些网络可以很好地近似估计其可见单元状态的任何概率分布。我们放宽了二进制单位的设置(Sutskever和Hinton, 2008 ; Le Roux&Bengio, 2008 2010 ;Montúfar&Ay, 2011 )用于具有任意有限状态空间的单位,并且消失的近似误差为任意的近似误差容限。例如,我们显示了具有 src =“ / ielx7 / 6720226/6855468/6855489 / abstract_images / NECO_a_00601inline1.gif” />“的 q 深度信任网络。 inline-formula>宽度为 src =“ / ielx7 / 6720226/6855468/6855489 / abstract_images / NECO_a_00601inline2.gif” /> 的图层,用于某些 src =“ / ielx7 / 6720226/6855468/6855489 / abstract_images / NECO_a_00601inline3.gif” /> 可以近似估计 src =“ / ielx7 / 6720226/6855468/6855489 / abstract_images / NECO_a_00601inline4.gif“ /> 不超过 src =” / ielx7 / 6720226/6855468/6855489 / abstract_images / NECO_a_00601inline5.gif“ /> < / inline-formula>。作为特殊情况,我们的分析涵盖离散的受限Boltzmann机器和朴素贝叶斯模型。

著录项

  • 来源
    《Neural computation》 |2014年第7期|1386-1407|共22页
  • 作者

    Montúfar G;

  • 作者单位

    Department of Mathematics, Pennsylvania State University, University Park, PA 16802, U.S.A. montufar@mis.mpg.de;

  • 收录信息 美国《科学引文索引》(SCI);美国《化学文摘》(CA);
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号