Distorted Low-Level Visual Features Affect Saliency-Based Visual Attention

摘要

Introduction Image distortions can attract attention away from the natural scene saliency (Redi et al., 2011 ). Performance of viewers in visual search tasks and their fixation patterns are also affected by different types and amounts of distortions (Vu et al., 2008 ). In this paper, we have discussed the opinion that distortions could largely affect the performance of predictive models of visual attention, and simulated the effects of distorted low-level visual features on the saliency-based bottom-up visual attention. Saliency is a fast and pre-attentive mechanism for orienting visual attention to intrinsically important objects which pop-out more easily in a cluttered scene. Distortion of the low-level features that contribute to saliency may impair the readiness of the visual system in detection of salient objects, which may have major implications for critical situations like driving or locomotion. These distortions in natural life can be introduced by eye diseases such as cataract, or spectacles which may alter color perception (de Fez et al., 2002 ) or cause undesired optical effects like blurring, non-uniform magnification, and image displacement (Barbero and Portilla, 2016 ). The extent to which each of these distorted saliency features may affect the attentional performance is addressed in this paper by employing a biologically-inspired predictive model of visual attention. We briefly summarize the current standing of computational work on visual attention models in the following section and suggest a simple and influential model of saliency to examine the above hypothesis. Furthermore, we demonstrate in an example the hindered performance of the predictive saliency model on distorted images. Models of visual attention Despite the widely shared belief in general public that we see everything around us, only a small fraction of the information registered by the visual system reaches levels of processing that mediate perception and directly influence behavior (Itti and Koch, 2000 ). Selective attention is the key to this process which turns looking into seeing (Carrasco, 2011 ). But how does the visual system select and enhance the representation of one particular feature or spatial location over less relevant features and locations? Much evidence has accumulated in favor of a two-component framework for the control of where in a visual scene attention is deployed: a bottom-up, fast, and image-based mechanism that biases the observer toward selecting stimuli based on their saliency, and a second slower, top-down mechanism, which uses task-dependent cues to direct the spotlight of attention under voluntary control. Koch and Ullman ( 1985 ) introduced the idea of a saliency map to accomplish pre-attentive selection. A saliency map is an explicit two-dimensional map that encodes the saliency of visual objects in the environment purely based on the low-level visual attributes of the object (Itti et al., 1998 ). Competition among neurons in this map gives rise to a single winning location that corresponds to the most salient object, which constitutes the next target. If this location is subsequently inhibited, the system automatically shifts to the next most salient location. This internal dynamic models the saccadic eye movements in visual search. This purely computational hypothesis received experimental support from many electrophysiological studies including single-cell recordings from lateral intraparietal neurons of macaque monkeys which responded to visual stimuli only when those stimuli were made salient (Gottlieb et al., 1998 ). Today, more than fifty quantitative models for saliency and fixation prediction are available which have been accumulated over the past 20 years; some of them tried to also incorporate top-down attention (Borji and Itti, 2013 ; Borji et al., 2013 ) or context-aware saliency detection (Goferman et al., 2012 ). However, not all of them are biologically plausible (Zhang and Sclaroff, 2013 ) or explain low-level visual features (Kümmerer et al., 2014 ); and the metrics used to compare the performance of these models are often different and inconsistent with each other (Kümmerer et al., 2015 ; Gide et al., 2016 ). For the purpose of this paper, we chose the original model of saliency-based visual attention for rapid scene analysis by Itti et al. ( 1998 ) for its utmost biological plausibility and simple dichotomy between low-level visual features which allows us to look at different distortion effects on each feature map independently, as well as the final saliency map. In the next section, we explain how this model can be used to simulate the effect of distortions on visual attention and visual search. Attention and image distortions Visual attention models have been used in many computer vision applications (Pal, 2016 ), including image and video compression and retrieval (Ouerhani et al., 2001 ; Li et al., 2011 ), multimedia technologies (Le Callet and Niebur, 2013 ), and