Synaptic efficiency modulations for context integration: The meta ODWE architecture

摘要

The new neural network architecture named ODWE (Orthogonal Delta Weight Estimator) implements a context dependent behavior by dynamically estimating the synaptic weight variations of a main MLP with regard to context parameters. The main MLP is first trained to modelize the general behavior of the task and then these connections are modulated to adapt to contextual parameters. We present in this paper a new extension of this principle by the generalization of the use of contextual modulation and show its performances in a combinatory problem: the multiplexor problem. We compare these results with other works.