MODELLING LETTER PERCEPTION: THE EFFECT OF SUPERVISION AND TOP-DOWN INFORMATION ON SIMULATED REACTION TIMES

摘要

In this study, we model human letter-recognition times using neural networks that extract visual features from real images of the letters. We focus on learning, and on how different learning methods and other factors affect the correlation between simulated reaction times and behavioural data. Specifically, we are interested in studying the effect of 3 factors on this correlation: (i) utilisation of an error signal during learning (supervised vs. unsupervised learning), (ii) whether or not the letter labels exert a top-down influence on the extracted features, and (iii) the effect of letter frequencies. To do so, we used Restricted Boltz- mann Machines (RBMs), Back-propagation networks, and RBM/Perceptron hybrid architectures. We find the highest correlations (r = 0.67) with supervised models when using top-down information of letter labels on the feature layer during training, but only when the letters' frequencies are taken into account during learning. This study shows that to account for human letter identification times, letter frequency seems to be the most important factor. In addition, top down information of letter labels on the extracted visual features appears to be essential (making the difference between a significant and nonsignificant correlation). Whether or not the model is supervised makes little difference in the correlation to human reaction time data, but fully unsuper- vised models have more difficulty generating accurate categorisation for letters with very low frequencies.