受限玻尔兹曼机的新混合稀疏惩罚机制

摘要

为解决受限玻尔兹曼机（RB M ）在学习过程中出现的特征同质化问题，在RB M 已有的稀疏模型基础上提出新的混合稀疏惩罚机制（HSPM ）。鉴于隐单元之间存在的统计相关性，该机制通过在RBM 训练过程中引入交叉熵稀疏惩罚因子，实现对RBM 的初步处理；按照基于 RBM 连接权值列相似性的自适应分组策略，构建稀疏组RBM ，并按照稀疏组受限玻尔兹曼机（SGRBM ）的形式继续进行隐单元稀疏化。实验结果表明：HSPM 能够有效解决RBM特征同质化问题，在隐单元的稀疏程度上优于以往的稀疏惩罚因子，可以整体提高RBM的特征提取能力，并可以成功应用于深度玻尔兹曼机（DBM ）的训练。%A new hybrid sparse penalty mechanism ( HSPM ) was proposed to resolve the features homogenization problem of restricted Boltzmann machine (RBM ) .HSPM was based on the existing sparse restricted Boltzmann machine (SRBM ) . Since the statistical correlation among hidden units , a cross‐entropy factor to optimize the training of RBMs was first implemented by HSPM .Then ,hidden units were grouped according to adaptive grouping strategy based on the column similarity of connection weights . Finally ,hidden units sparse processing was carried out in the form of sparse group restricted Boltzmann machine (SGRBM ) .The experimental results confirmed that HSPM could effectively resolve the feature homogenization problem of RBM and was better than ever sparse penalty factor on degree of signals sparsity .HSPM can improve the feature extraction capability of RBM and be applied to the training of deep Boltzmann machine (DBM ) successfully .