Understanding how people comprehend visual narratives (including picture stories, comics, and film) requires the combination of traditionally separate theories that span the initial sensory and perceptual processing of complex visual scenes, the perception of events over time, and comprehension of narratives. Existing piecemeal approaches fail to capture the interplay between these levels of processing. Here, we propose the Scene Perception & Event Comprehension Theory (SPECT), as applied to visual narratives, which distinguishes between front-end and (...) back-end cognitive processes. Front-end processes occur during single eye fixations and are comprised of attentional selection and information extraction. Back-end processes occur across multiple fixations and support the construction of event models, which reflect understanding of what is happening now in a narrative (stored in working memory) and over the course of the entire narrative (stored in long-term episodic memory). We describe relationships between front- and back-end processes, and medium-specific differences that likely produce variation in front-end and back-end processes across media (e.g., picture stories vs. film). We describe several novel research questions derived from SPECT that we have explored. By addressing these questions, we provide greater insight into how attention, information extraction, and event model processes are dynamically coordinated to perceive and understand complex naturalistic visual events in narratives and the real world. (shrink)

Eye movements during mental imagery are not epiphenomenal but assist the process of image generation. Commands to the eyes for each fixation are stored along with the visual representation and are used as spatial index in a motor‐based coordinate system for the proper arrangement of parts of an image. In two experiments, subjects viewed an irregular checkerboard or color pictures of fish and were subsequently asked to form mental images of these stimuli while keeping their eyes open. During the (...) perceptual phase, a group of subjects was requested to maintain fixation onto the screen's center, whereas another group was free to inspect the stimuli. During the imagery phase, all of these subjects were free to move their eyes. A third group of subjects (in Experiment 2) was free to explore the pattern but was requested to maintain central fixation during imagery. For subjects free to explore the pattern, the percentage of time spent fixating a specific location during perception was highly correlated with the time spent on the same (empty) locations during imagery. The order of scanning of these locations during imagery was correlated to the original order during perception. The strength of relatedness of these scanpaths and the vividness of each image predicted performance accuracy. Subjects who fixed their gaze centrally during perception did the same spontaneously during imagery. Subjects free to explore during perception, but maintaining central fixation during imagery, showed decreased ability to recall the pattern. We conclude that the eye scanpaths during visual imagery reenact those of perception of the same visual scene and that they play a functional role. (shrink)

Scene Perception is the visual perception of an environment as viewed by an observer at any given time. It includes not only the perception of individual objects, but also such things as their relative locations, and expectations about what other kinds of objects might be encountered. -/- Given that scene perception is so effortless for most observers, it might be thought of as something easy to understand. However, the amount of effort required by (...) a process often bears little relation to its underlying complexity. A closer look shows that scene perception is a highly complex activity, and that any account of it must deal with several difficult issues: What exactly is a scene? What aspects of it do we represent? And what are the processes involved? Finding the answers to these questions has proven to be extraordinarily difficult. (shrink)

This article presents two studies that deepen the theme of how soundtracks shape our interpretation of audiovisuals. Embracing a multivariate perspective, Study 1 ( N = 118) demonstrated, through an online between-subjects experiment, that two different music scores (melancholic vs. anxious) deeply affected the interpretations of an unknown movie scene in terms of empathy felt toward the main character, impressions of his personality, plot anticipations, and perception of the environment of the scene. With the melancholic music, participants (...) felt empathy toward the character, viewing him as more agreeable and introverted, more oriented to memories than to decisions, while perceiving the environment as cozier. An almost opposite pattern emerged with the anxious music. In Study 2 ( N = 92), we replicated the experiment in our lab but with the addition of eye-tracking and pupillometric measurements. Results of Study 1 were largely replicated; moreover, we proved that the anxious score, by increasing the participants’ vigilance and state of alert (wider pupil dilation), favored greater attention to minor details, as in the case of another character who was very hard to be noticed (more time spent on his figure). Results highlight the pervasive nature of the influence of music within the process of interpretation of visual scenes. (shrink)

People can discriminate the synchrony between audio-visual scenes. However, the sensitivity of audio-visual synchrony perception can be affected by many factors. Using a simultaneity judgment task, the present study investigated whether the synchrony perception of complex audio-visual stimuli was affected by audio-visual causality and stimulus reliability. In Experiment 1, the results showed that audio-visual causality could increase one's sensitivity to audio-visual onset asynchrony (AVOA) of both action stimuli and speech stimuli. Moreover, participants (...) were more tolerant of AVOA of speech stimuli than that of action stimuli in the high causality condition, whereas no significant difference between these two kinds of stimuli was found in the low causality condition. In Experiment 2, the speech stimuli were manipulated with either high or low stimulus reliability. The results revealed a significant interaction between audio-visual causality and stimulus reliability. Under the low causality condition, the percentage of “synchronous” responses of audio-visual intact stimuli was significantly higher than that of visual_intact/auditory_blurred stimuli and audio-visual blurred stimuli. In contrast, no significant difference among all levels of stimulus reliability was observed under the high causality condition. Our study supported the synergistic effect of top-down processing and bottom-up processing in audio-visual synchrony perception. (shrink)

Debates surrounding the high-level contents of perceptual experience focus on whether weperceive the high-level properties of visual objects, such as the property of being a pine tree. Thispaper considers instead whether we perceive the high-level properties of visual scenes, such asthe property of being a forest. Liberals about the contents of perceptual experience have offered avariety of phenomenal contrast cases designed to reveal how the high-level properties of objectsfigure in our visual experience. I offer a series of (...) equivalent phenomenal contrast cases intendedto show how the high-level properties of visual scenes also figure in visual experience. Thisfirst-person evidence of high-level scene perception is combined with third-person evidence fromthe extensive empirical literature on scene categorisation. Critics of liberalism have attempted todeflate existing phenomenal contrast cases by explaining the contrasts in terms of non-perceptualcontents or in terms of attentional changes. I show that neither response is applicable to mycontrast cases and conclude that we do indeed perceptually experience the high-level propertiesof visual scenes. (shrink)

Perception and action are tightly coupled. However, there is still little recognition of how individual motor constraints impact perception in everyday life. Here we asked whether and how the motor slowing that accompanies aging influences the sense of visual speed. Ninety-four participants aged between 18 and 90 judged the natural speed of video clips reproducing real human or physical motion. They also performed a finger tapping task and a visual search task, which estimated their motor speed (...) and visuospatial attention speed, respectively. Remarkably, aged people judged videos to be too slow, as compared to younger people: the Point of Subjective Equality, which estimated the speed bias in the SoS task, was +4% in young adults, +12% in old adults and +16% in elders. On average, PSE increased with age at a rate of 0.2% per year, with perceptual precision, adjustment rate, and completion time progressively worsening. Crucially, low motor speed, but not low attentional speed, turned out to be the key predictor of video speed underestimation. These findings suggest the existence of a counterintuitive compensatory coupling between action and perception in judging dynamic scenes, an effect that becomes particularly germane during aging. (shrink)

The theoretical notion of ‘construal’ captures the idea that the way in which we describe a scene reflects our conceptualization of it. Relying on the concept of ception – which conjoins conception and perception – we operationalized construal and employed a Visual World Paradigm to establish which aspects of linguistic scene description modulate visual scene perception, thereby affecting event conception. By analysing viewing behaviour after alternating ways of describing location (prepositions), agentivity (active/passive voice) (...) and transfer (NP/pp datives), we found that the linguistic construal of a scene affects its spontaneous visual perception in two ways: either by determining the order in which the components of a scene are accessed or by modulating the distribution of attention over the components, making them more or less salient than they naturally are. We also found evidence for the existence of a cline in the construal effect with stronger expressive differences, such as the prepositional manipulation, inducing more prominent changes in visual perception than the dative manipulation. We discuss the claims language can lay to affecting visual information uptake and hence conceptualization of a static scene in the light of these results. (shrink)

When referring to a target object in a visual scene, speakers are assumed to consider certain distractor objects to be more relevant than others. The current research predicts that the way in which speakers come to a set of relevant distractors depends on how they perceive the distance between the objects in the scene. It reports on the results of two language production experiments, in which participants referred to target objects in photo-realistic visual scenes. Experiment 1 (...) manipulated three factors that were expected to affect perceived distractor distance: two manipulations of perceptual grouping (region of space and type similarity), and one of presentation mode (2D vs. 3D). In line with most previous research on visually-grounded reference production, an offline measure of visual attention was taken here: the occurrence of overspecification with color. The results showed effects of region of space and type similarity on overspecification, suggesting that distractors that are perceived as being in the same group as the target are more often considered relevant distractors than distractors in a different group. Experiment 2 verified this suggestion with a direct measure of visual attention, eye tracking, and added a third manipulation of grouping: color similarity. For region of space in particular, the eye movements data indeed showed patterns in the expected direction: distractors within the same region as the target were fixated more often, and longer, than distractors in a different region. Color similarity was found to affect overspecification with color, but not gaze duration or the number of distractor fixations. Also the expected effects of presentation mode (2D vs. 3D) were not convincingly borne out by the data. Taken together, these results provide direct evidence for the close link between scene perception and language production, and indicate that perceptual grouping principles can guide speakers in determining the distractor set during reference production. (shrink)

I aim to give a new account of picture perception: of the way our visual system functions when we see something in a picture. My argument relies on the functional distinction between the ventral and dorsal visual subsystems. I propose that it is constitutive of picture perception that our ventral subsystem attributes properties to the scene, whereas our dorsal subsystem attributes properties to the surface. Keywords: picture perception; dorsal subsystem;.

Recent developments in vision science have resulted in several major changes in our understanding of human visual perception. For example, attention no longer appears necessary for "visual intelligence"--a large amount of sophisticated processing can be done without it. Scene perception no longer appears to involve static, general-purpose descriptions, but instead may involve dynamic representations whose content depends on the individual and the task. And vision itself no longer appears to be limited to the production of (...) a conscious "picture"--it may also guide processes outside the conscious awareness of the observer. (shrink)

The task of visual search is to determine as rapidly as possible whether a target item is present or absent in a display. Rapidly detected items are thought to contain features that correspond to primitive elements in the human visual system. In previous theories, it has been assumed that visual search is based on simple two-dimensional features in the image. However, visual search also has access to another level of representation, one that describes properties in the (...) corresponding three-dimensional scene. Among these properties are three dimensionality and the direction of lighting, but not viewing direction. These findings imply that the parallel processes of early vision are much more sophisticated than previously assumed. (shrink)

What role do moment‐to‐moment comprehension processes play in visual attentional selection in picture stories? The current work uniquely tested the role of bridging inference generation processes on eye movements while participants viewed picture stories. Specific components of the Scene Perception and Event Comprehension Theory (SPECT) were tested. Bridging inference generation was induced by manipulating the presence of highly inferable actions embedded in picture stories. When inferable actions are missing, participants have increased viewing times for the immediately following (...) critical image (Magliano, Larson, Higgs, & Loschky, ). This study used eye‐tracking to test competing hypotheses about the increased viewing time: (a) Computational Load: inference generation processes increase overall computational load, producing longer fixation durations; (b) Visual Search: inference generation processes guide eye‐movements to pick up inference‐relevant information, producing more fixations. Participants had similar fixation durations, but they made more fixations while generating inferences, with that process starting from the fifth fixation. A follow‐up hypothesis predicted that when generating inferences, participants fixate scene regions important for generating the inference. A separate group of participants rated the inferential‐relevance of regions in the critical images, and results showed that these inferentially relevant regions predicted differences in other viewers’ eye movements. Thus, viewers’ event models in working memory affect visual attentional selection while viewing visual narratives. (shrink)

As many as two million people in the United Kingdom repeatedly see people, animals, and objects that have no objective reality. Hallucinations on the border of sleep, dementing illnesses, delirium, eye disease, and schizophrenia account for 90% of these. The remainder have rarer disorders. We review existing models of recurrent complex visual hallucinations (RCVH) in the awake person, including cortical irritation, cortical hyperexcitability and cortical release, top-down activation, misperception, dream intrusion, and interactive models. We provide evidence that these can (...) neither fully account for the phenomenology of RCVH, nor for variations in the frequency of RCVH in different disorders. We propose a novel Perception and Attention Deficit (PAD) model for RCVH. A combination of impaired attentional binding and poor sensory activation of a correct proto-object, in conjunction with a relatively intact scene representation, bias perception to allow the intrusion of a hallucinatory proto-object into a scene perception. Incorporation of this image into a context-specific hallucinatory scene representation accounts for repetitive hallucinations. We suggest that these impairments are underpinned by disturbances in a lateral frontal cortex–ventral visual stream system. We show how the frequency of RCVH in different diseases is related to the coexistence of attentional and visual perceptual impairments; how attentional and perceptual processes can account for their phenomenology; and that diseases and other states with high rates of RCVH have cholinergic dysfunction in both frontal cortex and the ventral visual stream. Several tests of the model are indicated, together with a number of treatment options that it generates. Key Words: Blindness; Charles Bonnet; cholinergic; cortical release; delirium; dementia; dream intrusion; hallucination; Perception and Attention Deficit (PAD) model; schizophrenia. (shrink)

Correspondence should be addressed to David A. Leopold [email protected] the viewing of certain patterns, widely known as ambiguous or puzzle figures, perception lapses into a sequence of spontaneous alternations, switching every few seconds between two or more visual interpretations of the stimulus. Although their nature and origin remain topics of debate, these stochastic switches are generally thought to be the automatic and inevitable consequence of viewing a pattern without a unique solution. We report here that in humans such (...) perceptual alternations can be slowed, and even brought to a standstill, if the visual stimulus is periodically removed from view. We also show, with a visual illusion, that this stabilizing effect hinges on perceptual disappearance rather than on actual removal of the stimulus. These findings indicate that uninterrupted subjective perception of an ambiguous pattern is required for the initiation of the brain-state changes underlying multistable vision.Visual perception involves coordination between sensory sampling of the world and active interpretation of the sensory data. Human perception of objects and scenes is normally stable and robust, but it falters when one is presented with patterns that are inherently ambiguous or contradictory. Under such conditions, vision lapses into a chain of continually alternating percepts, whereby a viable visual interpretation dominates for a few seconds and is then replaced by a rival interpretation. This multistable vision, or 'multistability', is thought to result from destabilization of fundamental visual mechanisms, and has offered valuable insights into how sensory patterns are actively organized and interpreted in the brain1, 2. Despite a great deal of recent research and interest in multistable perception, however, its neurophysiological underpinnings remain poorly understood. Physiological studies have suggested that disambiguation of ambiguous patterns. (shrink)

Speakers’ perception of a visual scene influences the language they use to describe it—which objects they choose to mention and how they characterize the relationships between them. We show that visual complexity can either delay or facilitate description generation, depending on how much disambiguating information is required and how useful the scene's complexity can be in providing, for example, helpful landmarks. To do so, we measure speech onset times, eye gaze, and utterance content in a (...) reference production experiment in which the target object is either unique or non-unique in a visual scene of varying size and complexity. Speakers delay speech onset if the target object is non-unique and requires disambiguation, and we argue that this reflects the cost of deciding on a high-level strategy for describing it. The eye-tracking data demonstrate that these delays increase when speakers are able to conduct an extensive early visual search, implying that when speakers scan too little of the scene early on, they may decide to begin speaking before becoming aware that their description is underspecified. Speakers’ content choices reflect the visual makeup of the scene—the number of distractors present and the availability of useful landmarks. Our results highlight the complex role of visual perception in reference production, showing that speakers can make good use of complexity in ways that reflect their visual processing of the scene. (shrink)

A fundamental question in the psychological sciences is the degree to which culture shapes core cognitive processes — perhaps none more foundational than how we perceive the world around us. A dramatic and oft-cited “case study” of culture’s power in this regard is the Müller-Lyer illusion, which depicts two lines of equal length but with arrowheads pointing either inward or outward, creating the illusion that one line is longer than the other. According to a line of research stretching back over (...) a century, depending on the society you were raised in (and how much carpentry you were exposed to), you may not see the illusion at all — an ambitious and influential research program motivating claims that seemingly basic aspects of visual processing may actually be “culturally evolved byproducts”. This Cultural Byproduct Hypothesis bears on foundational issues in the science, philosophy, and sociology of psychology, and remains popular today. Yet, here we argue that it is almost certainly false. We synthesize evidence from diverse fields which demonstrate that: (1) the illusion is not limited to humans, appearing in non-human animals from diverse ecologies; (2) the statistics of natural scenes are sufficient to capture the illusion; (3) the illusion does not require straight lines typical of carpentry (nor even any lines at all); (4) the illusion arises in sense modalities other than vision; and (5) the illusion arises even in congenitally blind subjects. Moreover, by reexamining historical data and ethnographic descriptions from the original case studies, we show that the evidence for cultural variation and its correlation with key cultural variables is in fact highly inconsistent, beset by questionable research practices, and misreported by later discussions. Together, these considerations undermine the most popular and dramatic example of cultural influence on perception. We further extend our case beyond this phenomenon, showing that many of these considerations apply to other visual illusions as well, including similarly implicated visual phenomena such as the Ebbinghaus, Ponzo, Poggendorf, and Horizontal-Vertical illusions. We conclude by outlining future approaches to cross-cultural research on perception, and we also point to other potential sources of cultural variation in visual processing. (shrink)

Three of the most robust functional landmarks in the human brain are the selective responses to faces in the fusiform face area (FFA), scenes in the parahippocampal place area (PPA), and bodies in the extrastriate body area (EBA). Are the selective responses of these regions present early in development or do they require many years to develop? Prior evidence leaves this question unresolved. We designed a new 32-channel infant magnetic resonance imaging (MRI) coil and collected high-quality functional MRI (fMRI) data (...) from infants (2–9 months of age) while they viewed stimuli from four conditions—faces, bodies, objects, and scenes. We find that infants have face-, scene-, and body-selective responses in the location of the adult FFA, PPA, and EBA, respectively, powerfully constraining accounts of cortical development. (shrink)

When looking at a scene, observers feel that they see its entire structure in great detail and can immediately notice any changes in it. However, when brief blank fields are placed between alternating displays of an original and a modified scene, a striking failure of perception is induced: identification of changes becomes extremely difficult, even when changes are large and made repeatedly. Identification is much faster when a verbal cue is provided, showing that poor visibility is not (...) the cause of this difficulty. Identification is also faster for objects mentioned in brief verbal descriptions of the scene. These results support the idea that observers never form a complete, detailed representation of their surroundings. In addition, results also indicate that attention is required to perceive change, and that in the absence of localized motion signals it is guided on the basis of high-level interest. (shrink)

Selective access and integration underlie much of our visual experience. This article describes several of the experimental techniques used to investigate these processes, and some of the major results achieved in our understanding of their operation.

There has been a long-standing debate in philosophy and psychology about the role of representation in visual perception. Here, we argue on the basis of evidence from philosophy, psychology, and neuroscience that episodic and schematic memory representations are pivotal to the visual perception of objects and scenes. In the visual perception of objects and scenes, sensory information is initially matched with object and scene templates, or schemas, in long-term memory. The most relevant representations (...) are then selected for encoding in working memory. We furthermore argue that activations of episodic memory representations contribute to the fineness of grain of visual representations. The representational view of visual perception that emerges is what we call the "template tuning view." According to this view, prior information – specifically, long-term memories – shape the representational content of visual perception. In the final section of the chapter, we argue that unlike representational conceptions of visual perception, naïve and direct realist theories have difficulties accommodating these findings. (shrink)

The question of whether the visual world is a grand illusion is addressed and answered negatively. The question only arises because of the recent work on Inattentional Blindness , Change Blindness and the Attentional Blink which establishes that attention is necessary for perception. It is argued that IB occurs only when attention is narrowly focussed and not when attention is more broadly distributed, which is the more typical attentional state. Under conditions of distributed attention we are likely to (...) have a fuller, if less detailed, impression of the visual scene, which may be why we are so surprised by demonstrations of IB, AB and CB. It is also argued that the question about the possible illusory quality of our perceptual world cannot be avoided by denying that inattention causes blindness and asserting instead that it causes amnesia. This argument is grounded on the similarity between these phenomena and visual neglect and by evidence that priming by the unseen stimuli occurs in each case indicating that the stimuli to which we are functionally blind are processed and represented in implicit memory. The adaptive utility of this information is discussed. (shrink)

Eye movements were recorded during the display of two images of a real-world scene that were inspected to determine whether they were the same or not . In the displays where the pictures were different, one object had been changed, and this object was sometimes taken from another scene and was incongruent with the gist. The experiment established that incongruous objects attract eye fixations earlier than the congruous counterparts, but that this effect is not apparent until the picture (...) has been displayed for several seconds. By controlling the visual saliency of the objects the experiment eliminates the possibility that the incongruency effect is dependent upon the conspicuity of the changed objects. A model of scene perception is suggested whereby attention is unnecessary for the partial recognition of an object that delivers sufficient information about its visual characteristics for the viewer to know that the object is improbable in that particular scene, and in which full identification requires foveal inspection. (shrink)

This chapter presents an overview of several recent developments in vision science, and outlines some of their implications for the management of visual attention in graphic displays. These include ways of sending attention to the right item at the right time, techniques to improve attentional efficiency, and possibilities for offloading some of the processing typically done by attention onto nonattentional mechanisms. In addition it is argued that such techniques not only allow more effective use to be made of (...) class='Hi'>visual attention, but also open up new possibilities for human-machine interaction. (shrink)

One of the more compelling beliefs about vision is that it is based on representations that are coherent and complete, with everything in the visual field described in great detail. However, changes made during a visual disturbance are found to be difficult to see, arguing against the idea that our brains contain a detailed, picture-like representation of the scene. Instead, it is argued here that a more dynamic, "just-in-time" representation is involved, one with deep similarities to the (...) way that users interact with external displays. It is further argued that these similarities can provide a basis for the design of intelligent display systems that can interact with humans in highly effective and novel ways. (shrink)

One of the more powerful impressions created by vision is that of a coherent, richly-detailed world where everything is present simultaneously. Indeed, this impression is so compelling that we tend to ascribe these properties not only to the external world, but to our internal representations as well. But results from several recent experiments argue against this latter ascription. For example, changes in images of real-world scenes often go unnoticed when made during a saccade, flicker, blink, or movie cut. This "change (...) blindness" provides strong evidence against the idea that our brains contain a picture-like representation of the scene that is everywhere detailed and coherent. (shrink)

Perception is a central means by which we come to represent and be aware of particulars in the world. I argue that an adequate account of perception must distinguish between what one perceives and what one's perceptual experience is of or about. Through capacities for visual completion, one can be visually aware of particular parts of a scene that one nevertheless does not see. Seeing corresponds to a basic, but not exhaustive, way in which one can (...) be visually aware of an item. I discuss how the relation between seeing and visual awareness should be explicated within a representational account of the mind. Visual awareness of an item involves a primitive kind of reference: one is visually aware of an item when one's visual perceptual state succeeds in referring to that particular item and functions to represent it accurately. Seeing, by contrast, requires more than successful visual reference. Seeing depends additionally on meta-semantic facts about how visual reference happens to be fixed. The notions of seeing and of visual reference are both indispensable to an account of perception, but they are to be characterized at different levels of representational explanation. (shrink)

There is a certain excitement in vision science concerning the idea of applying the tools of bayesian decision theory to explain our perceptual capacities. Bayesian models are thought to be needed to explain how the inverse problem of perception is solved, and to rescue a certain constructivist and Kantian way of understanding the perceptual process. Anticlimactically, I argue both that bayesian outlooks do not constitute good solutions to the inverse problem, and that they are not constructivist in nature. In (...) explaining how visual systems derive a single percept from underdetermined stimulation, orthodox versions of bayesian accounts encounter a problem. The problem shows that such accounts need to be grounded in Natural Scene Statistics (NSS), an approach that takes seriously the Gibsonian insight that studying perception involves studying the statistical regularities of the environment in which we are situated. Additionally, I argue that bayesian frameworks postulate structures that hardly rescue a constructivist way of understanding perception. Except for percepts, the posits of bayesian theory are not representational in nature. bayesian perceptual inferences are not genuine inferences. They are biased processes that operate over nonrepresentational states. (shrink)

Recently, a number of experiments have emphasized the degree to which subjects fail to detect large changes in visual scenes. This finding, referred to as “change blindness,” is often considered surprising because many people have the intuition that such changes should be easy to detect. Levin, Momen, Drivdahl, and Simons documented this intuition by showing that the majority of subjects believe they would notice changes that are actually very rarely detected. Thus subjects exhibit a metacognitive error we refer to (...) as “change blindness blindness.” Here, we test whether CBB is caused by a misestimation of the perceptual experience associated with visual changes and show that it persists even when the pre- and postchange views are separated by long delays. In addition, subjects overestimate their change detection ability both when the relevant changes are illustrated by still pictures, and when they are illustrated using videos showing the changes occurring in real time. We conclude that CBB is a robust phenomenon that cannot be accounted for by failure to understand the specific perceptual experience associated with a change. (shrink)

A modified visual search task demonstrates that humans are very good at resuming a search after it has been momentarily interrupted. This is shown by exceptionally rapid response time to a display that reappears after a brief interruption, even when an entirely different visual display is seen during the interruption and two different visual searches are performed simultaneously. This rapid resumption depends on the stability of the visual scene and is not due to display or (...) response anticipations. These results are consistent with the existence of an iterative hypothesis-testing mechanism that compares information stored in short-term memory (the perceptual hypothesis) with information about the display (the sensory pattern). In this view, rapid resumption occurs because a hypothesis based on a previous glance of the scene can be tested very rapidly in a subsequent glance, given that the initial hypothesis-generation step has already been performed. (shrink)

It has often been assumed that when we use vision to become aware of an object or event in our surroundings, this must be accompanied by a corresponding visual experience (i.e., seeing). The studies reported here show that this assumption is incorrect. When observers view a sequence of displays alternating between an image of a scene and the same image changed in some way, they often feel (or sense) the change even though they have no visual experience (...) of it. The subjective difference between sensing and seeing is mirrored in several behavioral differences, suggesting that these are two distinct modes of conscious visual perception. (shrink)

This paper argues that a theory of situated vision, suited for the dual purposes of object recognition and the control of action, will have to provide something more than a system that constructs a conceptual representation from visual stimuli: it will also need to provide a special kind of direct (preconceptual, unmediated) connection between elements of a visual representation and certain elements in the world. Like natural language demonstratives (such as `this' or `that') this direct connection allows entities (...) to be referred to without being categorized or conceptualized. Several reasons are given for why we need such a preconcep- tual mechanism which individuates and keeps track of several individual objects in the world. One is that early vision must pick out and compute the relation among several individual objects while ignoring their properties. Another is that incrementally computing and updating representations of a dynamic scene requires keeping track of token individuals despite changes in their properties or locations. It is then noted that a mechanism meeting these requirements has already been proposed in order to account for a number of disparate empiri- cal phenomena, including subitizing, search-subset selection and multiple object tracking (Pylyshyn et al., Canadian Journal of Experimental Psychology 48(2) (1994) 260). This mechanism, called a visual index or FINST, is brie. (shrink)

Neuroscientists commonly assume that the brain generates representations of a scene in various non-retinotopic 3D coordinate frames, for example in 'egocentric' and 'allocentric' frames. Although neurons in early visual cortex might be described as representing a scene in an eye-centred frame, using 2 dimensions of visual direction and one of binocular disparity, there is no convincing evidence of similarly organized cortical areas using non-retinotopic 3D coordinate frames nor of any systematic transfer of information from one frame (...) to another. We propose that perception and action in a 3D world could be achieved without generating ego- or allocentric 3D coordinate frames. Instead, we suggest that the fundamental operation the brain carries out is to compare a long state vector with a matrix of weights to choose an output. The processes involved in perception of a 3D scene and action within it depend, we suggest, on successive iterations of this basic operation. Advantages of this proposal include the fact that it relies on computationally well-defined operations corresponding to well-established neural processes. Also, we argue that from a philosophical perspective it is at least as plausible as theories postulating 3D coordinate frames. Finally, we suggest a variety of experiments that would falsify our claim. (shrink)

Often implicit in visual display design and development is a gold standard of photorealism. By approximating direct perception, photorealism appeals to users and designers by being both attractive and apparently effortless. The vexing result from numerous performance evaluations, though, is that increasing realism often impairs performance. Smallman and St. John (2005) labeled misplaced faith in realistic information display Naïve Realism and theorized it resulted from a triplet of folk fallacies about perception. Here, we illustrate issues associated with (...) the wider trend towards realism by focusing on a specific current trend for high-fidelity perspective view (3D) geospatial displays. In two experiments, we validated Naïve Realism for different terrain understanding tasks, explored whether certain individuals are particularly prone to Naïve Realism, and determined the ability of task feedback to mitigate Naïve Realism. Performance was measured for laying and judging a concealed route across realistic terrain shown in different display formats. Task feedback was either implicit, in Experiment 1, or explicit in Experiment 2. Prospective and retrospective intuitions about the best display formats for the tasks were recorded and then related to task performance and participant spatial ability. Participants generally intuited they would perform tasks better with more realism than they actually required. For example, counter to intuitions, lowering fidelity of the terrain display revealed the gross scene layout needed to lay a well-concealed route. Individuals of high spatial ability calibrated their intuitions with only implicit task feedback, whereas those of low spatial ability required salient, explicit feedback to calibrate their intuitions about display realism. Results are discussed in the wider context of applying perceptual science to display design, and combating folk fallacies. (shrink)

When brief blank fields are placed between alternating displays of an original and a modified scene, a striking failure of perception is induced: the changes become extremely difficult to notice, even when they are large, presented repeatedly, and the observer expects them to occur (Rensink, O'Regan, & Clark, 1997). To determine the mechanisms behind this induced "change blindness", four experiments examine its dependence on initial preview and on the nature of the interruptions used. Results support the proposal that (...) representations at the early stages of visual processing are highly volatile, and that focused attention is needed to stabilize them sufficiently to support the perception of change. (shrink)

As Macnamara (1978) once asked, how can we talk about what we see? We report on a study manipulating realistic dynamic scenes and sentences aiming to understand the interaction between linguistic and visual representations in real-world situations. Specifically, we monitored participants’ eye movements as they watched video clips of everyday scenes while listening to sentences describing these scenes. We manipulated two main variables. The first was the semantic class of the verb in the sentence and the second was the (...) action/motion of the agent in the unfolding event. The sentences employed two verb classes—causatives (e.g., break) and perception/psychological (e.g., notice)—which impose different constraints on the nouns that serve as their grammatical complements. The scenes depicted events in which agents either moved towards a target object (always the referent of the verb-complement noun), away from it, or remained neutral performing a given activity (such as cooking). Scenes and sentences were synchronized such that the verb onset corresponded to the first video frame of the agent motion towards or away from the object. Results show effects of agent motion but weak verb-semantic restrictions: causatives draw more attention to potential referents of their grammatical complements than perception verbs only when the agent moves towards the target object. Crucially, we found no anticipatory verb-driven eye movements toward the target object, contrary to studies using non-naturalistic and static scenes. We propose a model in which linguistic and visual computations in real-world situations occur largely independent of each other during the early moments of perceptual input, but rapidly interact at a central, conceptual system using a common, propositional code. Implications for language use in real world contexts are discussed. (shrink)

Large changes in a scene often become difficult to notice if made during an eye movement, image flicker, movie cut, or other such disturbance. It is argued here that this _change blindness_ can serve as a useful tool to explore various aspects of vision. This argument centers around the proposal that focused attention is needed for the explicit perception of change. Given this, the study of change perception can provide a useful way to determine the nature of (...) visual attention, and to cast new light on the way that it is?and is not?involved in visual perception. To illustrate the power of this approach, this paper surveys its use in exploring three different aspects of vision. The first concerns the general nature of _seeing_. To explain why change blindness can be easily induced in experiments but apparently not in everyday life, it is proposed that perception involves a _virtual representation_, where object representations do not accumulate, but are formed as needed. An architecture containing both attentional and nonattentional streams is proposed as a way to implement this scheme. The second aspect concerns the ability of observers to detect change even when they have no visual experience of it. This _sensing_ is found to take on at least two forms: detection without visual experience (but still with conscious awareness), and detection without any awareness at all. It is proposed that these are both due to the operation of a nonattentional visual stream. The final aspect considered is the nature of visual attention itself?the mechanisms involved when _scrutinizing_ items. Experiments using controlled stimuli show the existence of various limits on visual search for change. It is shown that these limits provide a powerful means to map out the attentional mechanisms involved. (shrink)

Luminance and color are strong and self-sufficient cues to pictorial depth in visual scenes and images. The present study investigates the conditions Under which luminance or color either strengthens or overrides geometric depth cues. We investigated how luminance contrasts associated with color contrast interact with relative height in the visual field, partial occlusion, and interposition in determining the probability that a given figure is perceived as ‘‘nearer’’ than another. Latencies of ‘‘near’’ responses were analyzed to test for effects (...) of attentional selection. Figures in a pair were supported by luminance contrast or isoluminant color contrast and combined with one of the three geometric cues. The results of Experiment 1 show that luminance contrasts associated with hue, when it does not interact with other hues, produces the same effects as achromatic luminance contrasts: The probability of‘‘near’’ increases with luminance contrast while the latencies for ‘‘near’’ responses decrease. Partial occlusion is found to be a strong enough pictorial cue to support a weaker red luminance contrast. Interposition cues lose out against cues of spatial position and partial occlusion. The results of Experiment 2, with isoluminant displays of varying color contrast, reveal that red color contrast on a light background supported by any of the three geometric cues wins over green or white supported by any of the three geometric cues. On a dark background, red color contrast supported by the interposition cue loses out against green or white color contrast supported by partial occlusion. These findings reveal that color is not an independent depth cue, but is strongly influenced by luminance contrast and stimulus geometry. Systematically shorter response latencies for stronger ‘‘near’’ percepts demonstrate that selective visual attention reliably detects the most likely depth cue combination in a given configuration. (shrink)

A fundamental characteristic of human visual perception is the ability to group together disparate elements in a scene and treat them as a single unit. The mechanisms by which humans create such groupings remain unknown, but grouping seems to play an important role in a wide variety of visual phenomena. I propose a neural model of grouping; through top-down control of its circuits, the model implements a grouping strategy that involves both a connection strategy (which elements (...) to connect) and a selection strategy (spatiotemporal properties of a selection signal that segments target elements to facilitate identification). With computer simulations I explain how the circuits work and show how they can account for a wide variety of Gestalt principles of perceptual grouping. Additionally, I extend the model so that it can simulate visual search tasks. I show that when the model uses particular grouping strategies, simulated results closely match empirical results from replication experiments of three visual search tasks. In these experiments, perceptual grouping was induced by proximity and shape similarity (Palmer & Beck, 2007), by the spacing of irrelevant distractors and size similarity (Vickery, 2008), or by the proximity of dots and the proximity and shape similarity of line figures (Trick & Enns, 1997). Thus, I show that the model accounts for a variety of grouping effects and indicates which grouping strategies were likely used to promote performance in three visual search tasks. (shrink)

There is mounting evidence that language comprehension involves the activation of mental imagery of the content of utterances (; ; ; ; ; ; ). This imagery can have motor or perceptual content. Three main questions about the process remain under‐explored, however. First, are lexical associations with perception or motion sufficient to yield mental simulation, or is the integration of lexical semantics into larger structures, like sentences, necessary? Second, what linguistic elements (e.g., verbs, nouns, etc.) trigger mental simulations? Third, (...) how detailed are the visual simulations that are performed? A series of behavioral experiments address these questions, using a visual object categorization task to investigate whether up‐ or down‐related language selectively interferes with visual processing in the same part of the visual field (following ). The results demonstrate that either subject nouns or main verbs can trigger visual imagery, but only when used in literal sentences about real space—metaphorical language does not yield significant effects—which implies that it is the comprehension of the sentence as a whole and not simply lexical associations that yields imagery effects. These studies also show that the evoked imagery contains detail as to the part of the visual field where the described scene would take place. (shrink)

East Asians and Westerners differ in various aspects of perception and cognition. For example, visual memory for East Asians is believed to be more influenced by the contextual aspects of a scene than is the case for Westerners (Masuda & Nisbett, 2001). There are also differences in visual search: for Westerners, search for a long line among short is faster than for short among long, whereas this difference does not appear to hold for East Asians (Ueda (...) et al., submitted). However, it is unclear how these group-level differences originate. To investigate the extent to which they depend upon environment, we tested visual search and visual memory in East Asian immigrants who had lived in Canada for different amounts of time. Recent immigrants were found to exhibit no search asymmetry, unlike Westerners who had spent their lives in Canada. However, immigrants who had lived in Canada for more than two years showed performance comparable to that of Westerners. These differences could not be explained by the general analytic/holistic processing distinction believed to differentiate Westerners and East Asians, since all observers showed a strong holistic tendency for visual recognition. Results instead support the suggestion that exposure to a new environment can significantly affect the particular processes used to perceive a given stimulus. (shrink)