A vision of language: The mechanism, development and consequence of an interaction between language and vision.

摘要

Most researchers in the cognitive sciences assume that language and nonlinguistic cognition interact. For instance, language and vision interact, allowing us to talk about what we see. In fact close to 100 years of experimental research has shown that doing certain tasks in the presence of language (verbal instructions or verbal labels) is different from doing the same tasks in the absence of language. Despite the abundance of such "language effects" in the literature, very little is known about the level at which language has an effect (does it affect input, computation, and/or representation), what aspect of language is crucial for the observed effect (syntax, semantic and/or phonology), and the mechanism by which language has an effect. In this dissertation, I examine the mechanism, development and representational consequence of the interaction between language and vision by focusing on a specific case of a language-vision interaction.;Vision researchers have shown that binding visual features (e.g., color and shape) and maintaining these bound features in memory is difficult for adults and children. For example, if people are briefly presented with a display containing red Os and green Ls, they will mistakenly report that they have seen either a red L or a green O (e.g., Treisman & Schmidt, 1982). Dessalegn & Landau have shown that 4-year olds have difficulty maintaining in memory the conjunction of color and location. They have also shown that this difficulty is overcome when the visual target is accompanied by sentences containing directional asymmetric predicates (e.g., "See this? The red is on the left of the green"). Moreover they showed that simply labeling the target with novel labels (e.g., "See this? This is a dax") or sentences with symmetric predicates (e.g., "See this? The red is touching the green") did not help children remember the conjunction of color and location. Finally, they showed that nonlinguistic attentional manipulations (e.g., flashing on and off a part of the visual target) failed to help children form a stable representation of the color-location combination.;In this dissertation I propose the Representation Updating Hypothesis to account for the mechanism by which certain linguistic labels, but not others, helped memory for color-location conjunction. I test a prediction of the Representation Updating hypothesis. I also examine the development of the specific vision-language interaction described in Dessalegn & Landau (2008) by testing 3, 4, 6 year olds and adults. Finally, a Language-Vision Interaction Framework is proposed to account for the present findings and other findings in the literature.