Interpreting neural networks is a popular topic, and there are many works focusing on analyzing networks with respect to learning syntax (Shi et al., 2016; Linzen et al., 2016; Blevins et al., 2018).In particular, Vaswani et al. (2017) showed that the self-attentions in their Transformer architecture may be directly interpreted as syntactic dependencies between tokens. However, there is a potential problem in the fact that the attention mechanism on deeper layers operates on the previous-layer neurons, which already comprise mixed information from multiple source tokens.
展开▼
机译：解释神经网络是一个流行的主题，并且有许多作品专注于在学习语法上分析网络（Shi等，2016; Linzen等，2016; Blevins等，2018）。特别是，Vaswani et al。 （2017）表明，变压器架构中的自我关注可以直接被解释为令牌之间的句法依赖性。然而，在更深层上的注意机制在前一层神经元上操作的事实上存在潜在的问题，该神经元已经包括来自多个源代币的混合信息。
展开▼
