首页> 外文期刊>IEEE Transactions on Knowledge and Data Engineering >Natural language grammatical inference with recurrent neural networks
【24h】

Natural language grammatical inference with recurrent neural networks

机译:递归神经网络的自然语言语法推断

获取原文
获取原文并翻译 | 示例

摘要

This paper examines the inductive inference of a complex grammar with neural networks and specifically, the task considered is that of training a network to classify natural language sentences as grammatical or ungrammatical, thereby exhibiting the same kind of discriminatory power provided by the Principles and Parameters linguistic framework, or Government-and-Binding theory. Neural networks are trained, without the division into learned vs. innate components assumed by Chomsky (1956), in an attempt to produce the same judgments as native speakers on sharply grammatical/ungrammatical data. How a recurrent neural network could possess linguistic capability and the properties of various common recurrent neural network architectures are discussed. The problem exhibits training behavior which is often not present with smaller grammars and training was initially difficult. However, after implementing several techniques aimed at improving the convergence of the gradient descent backpropagation-through-time training algorithm, significant learning was possible. It was found that certain architectures are better able to learn an appropriate grammar. The operation of the networks and their training is analyzed. Finally, the extraction of rules in the form of deterministic finite state automata is investigated.
机译:本文研究了具有神经网络的复杂语法的归纳推理,具体而言,所考虑的任务是训练网络将自然语言句子分类为语法或非语法的类别,从而展现出“原则和参数”语言所提供的同类型歧视力。框架或政府约束理论。训练神经网络时,不将Chomsky(1956)假定的学习成分与先天成分进行区分,以试图在语法或非语法数据上得出与母语使用者相同的判断。讨论了递归神经网络如何具有语言能力以及各种常见的递归神经网络体系结构的属性。该问题表现出较小的语法通常不存在的训练行为,并且训练最初很困难。但是,在实施了几种旨在改善梯度下降反向传播时域训练算法的收敛性的技术之后,大量的学习是可能的。发现某些体系结构能够更好地学习适当的语法。分析了网络的运行及其培训。最后,研究了确定性有限状态自动机形式的规则提取。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号