Changing the information format from probabilities into frequencies as well as employing appropriate visualizations such as tree diagrams or 2 × 2 tables are important tools that can facilitate people’s statistical reasoning. Previous studies have shown that despite their widespread use in statistical textbooks, both of those visualization types are only of restricted help when they are provided with probabilities, but that they can foster insight when presented with frequencies instead. In the present study, we attempt to replicate this effect (...) and also examine, by the method of eye tracking, why probabilistic 2 × 2 tables and tree diagrams do not facilitate reasoning with regard to Bayesian inferences (i.e., determining what errors occur and whether they can be explained by scan paths), and why the same visualizations are of great help to an individual when they are combined with frequencies. All ten inferences of N = 24 participants were based solely on tree diagrams or 2 × 2 tables that presented either the famous “mammography context” or an “economics context” (without additional textual wording). We first asked participants for marginal, conjoint, and (non-inverted) conditional probabilities (or frequencies), followed by related Bayesian tasks. While solution rates were higher for natural frequency questions as compared to probability versions, eye-tracking analyses indeed yielded noticeable differences regarding eye movements between correct and incorrect solutions. For instance, heat maps (aggregated scan paths) of distinct results differed remarkably, thereby making correct and faulty strategies visible in the line of theoretical classifications. Moreover, the inherent structure of 2 × 2 tables seems to help participants avoid certain Bayesian mistakes (e.g., “Fisherian” error) while tree diagrams seem to help steer them away from others (e.g., “joint occurrence”). We will discuss resulting educational consequences at the end of the paper. (shrink)

This study investigates the online process of reading and analyzing of sketchnotes (visual notes containing a handwritten text and drawings) on Russian language material. Using the eye-tracking method, we compared the processing of different types of sketchnotes (‘path’ (trajectory), linear, and radial) and the processing of a verbal text. Biographies of Russian writers were used as the material. In a preliminary experiment, we asked 89 college students to read the biographies and to evaluate each text or sketch using five (...) scales (from -2 to +2). The best example for each of three formats of sketchnotes and a verbal text were chosen. In the main experiment, twenty-one secondary school students examined four different biographies in four different formats (three sketchnotes and a verbal text), answered to the factual and analytical questions to these texts and estimated the difficulty of each text. We measured the total dwell time, the total fixation count, the average fixation duration for each stimulus as well as for separate zones inside the sketches including verbal and non-verbal information. Our results show that readers process the information better and faster while reading sketchnotes than a verbal text. In the trajectory sketchnotes, the readers followed the order of elements aimed by the author of the sketchnotes better than in the radial and linear sketchnotes. The analysis of participants’ eye movements while processing the stimuli made it possible to propose several recommendations for creating effective sketchnotes. (shrink)

Failure increases the motivation to escape self-awareness. To date, however, the role of self-conscious emotions (shame and guilt) in triggering escape responses after failure has not been sufficiently addressed. In this pre-registered study (N = 156 undergraduates), we adapted a classic paradigm (avoidance of one’s image in a mirror) to a modern eye-tracking technology to test the hypothesis that shame proneness moderates the effect of failure on self-awareness avoidance. Individual differences in guilt and shame proneness were assessed before priming (...) thoughts of failure or success. Then, an eye-tracking paradigm was used to monitor gaze avoidance of one’s screen-reflected face during a neutral, unrelated task. Unexpectedly, results showed that guilt but not shame proneness exacerbated self-avoidance after failure. The present findings challenge the dominant view that shame fosters avoidance more so than guilt. (shrink)

Face perception and emotion categorization are widely investigated under laboratory conditions that are devoid of real social interaction. Using mobile eye-tracking glasses in a standardized diagnostic setting while applying the Autism Diagnostic Observation Schedule (ADOS-2), we had the opportunity to record gaze behavior of children and adolescents with and without Autism Spectrum Conditions (ASCs) during social interaction. The objective was to investigate differences in eye-gaze behavior between three groups of children and adolescents either (1) with ASC or (2) with (...) unconfirmed diagnosis of ASC or (3) with neurotypical development (NTD) during social interaction with an adult interviewer in a diagnostic standard situation using the ADOS-2. In a case control study, we used mobile eye-tracking glasses in an ecologically valid and highly standardized diagnostic interview to investigate suspected cases of ASC. After completion of the ASC diagnostic gold standard including the ADOS-2, the participants were assigned to two groups based on their diagnosis (ASC vs. non-ASC) and compared with a matched group of neurotypically developed controls. The primary outcome measure is the percentage of total dwell times assessed for different areas of interest (AOI) with regard to the face and body of a diagnostic interviewer and the surrounding space. Overall, 65 children and adolescents within an age range of 8.3–17.9 years were included in the study. The data revealed significant group differences, especially in the central-face area. Previous investigations under laboratory conditions gave preferential attention to the eye region during face perception to describe differences between ASC and NTD. In this study – using an ecologically valid setting within a standard diagnostic procedure – the results indicate that neurotypically developed controls seem to process faces and facial expressions in a holistic manner originating from the central-face region. Conversely, participants on the Autism Spectrum (tAS) seem to avoid the central-face region and show unsystematic gaze behavior, not using the preferred landing position in the central-face region as the Archimedean point of face perception. This study uses a new approach, and it will be important to replicate these preliminary findings in future research. (shrink)

Multimodal Critical Discourse Analysis (MCDA) as a sub-discipline of Critical Discourse Analysis (CDA) emerged from the availability of social semiotic frameworks describing multimodal meaning making. However, weaknesses of these frameworks have raised concerns and prompted recent methodological reflections in MCDA. Inspired by these reflections, this paper critically assesses MCDA research on advertising persuasion and identifies a lack of attention in studies to account for the social and ideological impact of advertising. This shortcoming is argued to be attributable to the weak (...) empirical grounding of social semiotic frameworks which in turn is rooted in a ‘discursive gap’ (Bateman, Citation2019, p. 300) in multimodality as a discipline. The authors then propose methodological solutions involving eye tracking experiments and higher order data analytics which can strengthen the empirical basis of theoretical assumptions made in MCDA research on advertising. Drawing on methodological guidance for empirical multimodal research, the authors discuss specific issues in realizing this proposal, focusing on a strategy addressing operationalization challenges in experimental design and the affordance of a data cluster algorithm that calculates the common viewing path of a group of viewers. (shrink)

Semantic studies on diagrammatic notations (Barwise & Etchemendy, ; Shimojima, ; Stenning & Lemon, ) have revealed that the “non-deductive,” “emergent,” or “perceptual” effects of diagrams (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, ; Kulpa, ; Larkin & Simon, ; Lindsay, ) are all rooted in the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the frequently observed reduction of inferential load. The purpose of this study was to examine (...) the empirical basis for this theoretical suggestion, focusing on the reality of the constraint-exploitation strategy in actual practices of diagrammatic reasoning. Eye movements were recorded while participants used simple position diagrams to solve three- or four-term transitive inference problems. Our experiments revealed that the participants could exploit spatial constraints on graphical structures even when (a) they were not in the position of actually manipulating diagrams, (b) the semantic rule for the provided diagrams did not match their preferences, and (c) the constraint-exploitation strategy invited a partly adverse effect. These findings indicate that the hypothesized process is in fact robust, with the potential to broadly account for the inferential advantage of diagrams. (shrink)

It has been four decades since eye-tracking was first used in interpreting studies, and recent years has witnessed a growing interest in the application of this method, which holds great potential for offering a look into the “black box” of interpreting processing. However, little attention has been paid to comprehensively illustrating what has been done, what can be done, and what needs to be done with this method in this discipline. With this in view, this paper sets out to (...) understand contributions of previous studies—key themes discussed, eye-tracking measures used, their limitations and implications, and future directions. To this end, we conduct a review of a total of 26 empirical papers from peer-reviewed journals within a time span of 4 decades ranging from 1981 to 2021. This study, as the first attempt of its kind at a comprehensive review on using eye-tracking in interpreting studies, should have implications for researchers, educators, and practitioners. (shrink)

Several recent studies have demonstrated context-based, high-confidence misperceptions in hearing, referred to as false hearing. These studies have unanimously found that older adults are more susceptible to false hearing than are younger adults, which the authors have attributed to an age-related decline in the ability to inhibit the activation of a contextually predicted response. However, no published work has investigated this activation-based account of false hearing. In the present study, younger and older adults listened to sentences in which the semantic (...) context provided by the sentence was either unpredictive, highly predictive and valid, or highly predictive and misleading with relation to a sentence-final word in noise. Participants were tasked with clicking on one of four images to indicate which image depicted the sentence-final word in noise. We used eye-tracking to investigate how activation, as revealed in patterns of fixations, of different response options changed in real-time over the course of sentences. We found that both younger and older adults exhibited anticipatory activation of the target word when highly predictive contextual cues were available. When these contextual cues were misleading, younger adults were able to suppress the activation of the contextually predicted word to a greater extent than older adults. These findings are interpreted as evidence for an activation-based model of speech perception and for the role of inhibitory control in false hearing. (shrink)

How do people evaluate causal relationships? Do they just consider what actually happened, or do they also consider what could have counterfactually happened? Using eye tracking and Gaussian process modeling, we investigated how people mentally simulated past events to judge what caused the outcomes to occur. Participants played a virtual ball‐shooting game and then—while looking at a blank screen—mentally simulated (a) what actually happened, (b) what counterfactually could have happened, or (c) what caused the outcome to happen. Our findings (...) showed that participants moved their eyes in patterns consistent with the actual or counterfactual events that they mentally simulated. When simulating what caused the outcome to occur, participants moved their eyes consistent with simulations of counterfactual possibilities. These results favor counterfactual theories of causal reasoning, demonstrate how eye movements can reflect simulation during this reasoning and provide a novel approach for investigating retrospective causal reasoning and counterfactual thinking. (shrink)

This exploratory study aims at integrating the psychometric approach to studying creativity with an eye-tracking methodology and thinking-aloud protocols to potentially untangle the nuances of the creative process. Wearing eye-tracking glasses, one hundred adults solved a drawing creativity test – The Test of Creative Thinking-Drawing Production (TCT-DP) – and provided spontaneous comments during this process. Indices of visual activity collected during the eye-tracking phase explained a substantial amount of variance in psychometric scores obtained in the test. More (...) importantly, however, clear signs of methodological synergy were observed when all three sources (psychometrics, eye-tracking, and coded thinking-aloud statements) were integrated. The findings illustrate benefits of using a blended methodology for a more insightful analysis of creative processes, including creative learning and creative problem-solving. (shrink)

The aim of this study was to find out how people process the dialectal variation encountered in the daily linguistic input. We conducted an eye tracking study (Visual Word Paradigm) that targeted the predictive processing of grammatical gender markers. Three different groups of Norwegian speakers took part in the experiment: one group of students from the capital Oslo, and two groups of dialect speakers from the Western Norwegian town Sogndal. One Sogndal group was defined as ``stable dialect speakers'', and (...) one as ``unstable dialect speakers'', based on a background questionnaire. The students participated in two eye tracking experiments each, one conducted in the their own dialect, and one in the other dialect (i.e., Sogn dialect for the Oslo students, and Oslo dialect for the Sogndal students). The gender systems in the two dialects differ: the Sogn dialect makes an obligatory three-gender split (Masculine, Feminine and Neuter) whereas the Oslo dialect only obligatorily makes a two gender distinction. The research question was whether speakers could use gender markers to predict the upcoming target noun in both local and non-local dialect mode, and furthermore, if they correctly could adjust their expectations based on dialect mode. The results showed that the Sogn speakers could predict upcoming linguistic material both in the local and Oslo dialect, but only the stable group were able to adjust their predictions based on the dialect mode. The unstable group applied a more general Oslo-compatible parsing to both the local and the non-local dialect. The Oslo speakers on the other hand were able to use gender markers as predictors only in their own dialect. We argue that the stable Sogn group should be treated as a bilingual group, as they show native-like skills in both varieties, while the unstable Sogn group can be seen as accommodated monolinguals, in that they treat the two varieties as sharing an underspecified grammar. The Oslo group on the other hand lacks sufficient competence in the other dialect to make use of grammatical markers to make predictions. (shrink)

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, and minimal volitional movement (...) completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: commercially available eye tracking communication software, Shuffle Speller with modified eye tracking, and Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants’ typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed. (shrink)

Background Visual atypicalities in autism spectrum disorder (ASD) are a well documented phenomenon, beginning as early as 2–6 months of age and manifesting in a significantly decreased attention to the eyes, direct gaze and socially salient information. Early emerging neurobiological deficits in perceiving social stimuli as rewarding or its active avoidance due to the anxiety it entails have been widely purported as potential reasons for this atypicality. Parallel research evidence also points to the significant benefits of animal presence for reducing (...) social anxiety and enhancing social interaction in children with autism. While atypicality in social attention in ASD has been widely substantiated, whether this atypicality persists equally across species types or is confined to humans has not been a key focus of research insofar. Methods We attempted a comprehensive examination of the differences in visual attention to static images of human and animal faces (40 images; 20 human faces and 20 animal faces) among children with ASD using an eye tracking paradigm. 44 children (ASD n = 21; TD n = 23) participated in the study (10,362 valid observations) across five regions of interest (left eye, right eye, eye region, face and screen). Results Results obtained revealed significantly greater social attention across human and animal stimuli in typical controls when compared to children with ASD. However in children with ASD, a significantly greater attention allocation was seen to animal faces and eye region and lesser attention to the animal mouth when compared to human faces, indicative of a clear attentional preference to socially salient regions of animal stimuli. The positive attentional bias toward animals was also seen in terms of a significantly greater visual attention to direct gaze in animal images. Conclusion Our results suggest the possibility that atypicalities in social attention in ASD may not be uniform across species. It adds to the current neural and biomarker evidence base of the potentially greater social reward processing and lesser social anxiety underlying animal stimuli as compared to human stimuli in children with ASD. (shrink)

Unilateral spatial neglect is a disorder characterized by the failure to report, respond to, or orient toward the contralateral side of space to a brain lesion. Current assessment methods often fail to discover milder forms, cannot differentiate between unilateral spatial neglect subtypes and lack ecological validity. There is also a need for treatment methods that target subtypes. Immersive virtual reality systems in combination with eye-tracking have the potential to overcome these shortcomings, by providing more naturalistic environments and tasks, with (...) sensitive and detailed measures. This systematic review examines the state of the art of research on these technologies as applied in the assessment and treatment of USN. As we found no studies that combined immersive VR and ET, we reviewed these approaches individually. The review of VR included seven articles, the ET review twelve. The reviews revealed promising results. All included studies found significant group-level differences for several USN measures. In addition, several studies found asymmetric behavior in VR and ET tasks for patients who did not show signs of USN in conventional tests. Particularly promising features were multitasking in complex VR environments and detailed eye-movement analysis. No VR and only a few ET studies attempted to differentiate USN subtypes, although the technologies appeared appropriate. One ET study grouped USN participants using individual heatmaps, and another differentiated between subtypes on drawing tasks. Regarding ecological validity, although no studies tested the prognostic validity of their assessment methods, VR and ET studies utilized naturalistic tasks and stimuli reflecting everyday situations. Technological characteristics, such as the field of view and refresh rate of the head-mounted displays, could be improved, though, to improve ecological validity. We found no studies that utilized VR or ET technologies for USN treatment up until the search date of the 26th of February 2020. In conclusion, VR-ET-based systems show great potential for USN assessment. VR-ET holds great promise for treatment, for example, by monitoring behavior and adapting and tailoring to the individual person’s needs and abilities. Future research should consider developing methods for individual subtypes and differential diagnostics to inform individual treatment programs. (shrink)

Word order alternation has been described as one of the most productive information structure markers and discourse organizers across languages. Psycholinguistic evidence has shown that word order is a crucial cue for argument interpretation. Previous studies about Spanish sentence comprehension have shown greater difficulty to parse sentences that present a word order that does not respect the order of participants of the verb's lexico-semantic structure, irrespective to whether the sentences follow the canonical word order of the language or not. This (...) difficulty has been accounted as the cognitive cost related to the miscomputation of prominence status of the argument that precedes the verb. Nonetheless, the authors only analyzed the use of alternative word orders in isolated sentences, leaving aside the pragmatic motivation of word order alternation. By means of an eye-tracking task, the current study provides further evidence about the role of information structure for the comprehension of sentences with alternative word order and verb type, and sheds light on the interaction between syntax, semantics and pragmatics. We analyzed both “early” and “late” eye-movement measures as well as accuracy and response times to comprehension questions. Results showed an overall influence of information structure reflected in a modulation of late eye-movement measures as well as offline measures like total reading time and questions response time. However, effects related to the miscomputation of prominence status did not fade away when sentences were preceded by a context that led to non-canonical word order of constituents, showing that prominence computation is a core mechanism for argument interpretation, even in sentences preceded by context. (shrink)

The visual behaviour of consumers buying products in a supermarket was measured and used to analyse the stages of their decision process. Traditionally metrics used to trace decision-making processes are difficult to use in natural environments that often contain many options and unstructured information. Unlike previous attempts in this direction, our methodology reveals differences between a decision-making task and a search task. In particular the second stage of a decision task contains more re-dwells than the second stage of a comparable (...) search task. This study addresses the growing concern of taking eye movement research from the laboratory into the ‘real-world’, so findings can be better generalised to natural situations. (shrink)

Difficulties understanding speech form one of the most prevalent complaints among older adults. Successful speech perception depends on top-down linguistic and cognitive processes that interact with the bottom-up sensory processing of the incoming acoustic information. The relative roles of these processes in age-related difficulties in speech perception, especially when listening conditions are not ideal, are still unclear. In the current study, we asked whether older adults with a larger working memory capacity process speech more efficiently than peers with lower capacity (...) when speech is presented in noise, with another task performed in tandem. Using the Eye-tracking of Word Identification in Noise Under Memory Increased Load an adapted version of the “visual world” paradigm, 36 older listeners were asked to follow spoken instructions presented in background noise, while retaining digits for later recall under low or high memory load. In critical trials, instructions directed listeners’ gaze to pictures of objects whose names shared onset or offset sounds with the name of a competitor that was displayed on the screen at the same time. We compared listeners with different memory capacities on the time course for spoken word recognition under the two memory loads by testing eye-fixations on a named object, relative to fixations on an object whose name shared phonology with the named object. Results indicated two trends. For older adults with lower working memory capacity, increased memory load did not affect online speech processing, however, it impaired offline word recognition accuracy. The reverse pattern was observed for older adults with higher working memory capacity: increased task difficulty significantly decreases online speech processing efficiency but had no effect on offline word recognition accuracy. Results suggest that in older adults, adaptation to adverse listening conditions is at least partially supported by cognitive reserve. Therefore, additional cognitive capacity may lead to greater resilience of older listeners to adverse listening conditions. The differential effects documented by eye movements and accuracy highlight the importance of using both online and offline measures of speech processing to explore age-related changes in speech perception. (shrink)

Experimental aesthetics has shed light on the involvement of pre-motor areas in the perception of abstract art. However, the contribution of texture perception to aesthetic experience is still understudied. We hypothesized that digital screen-based art, despite its immateriality, might suggest potential sensorimotor stimulation. Original born-digital works of art were selected and manipulated by the artist himself. Five behavioral parameters: Beauty, Liking, Touch, Proximity, and Movement, were investigated under four experimental conditions: Resolution (high/low), and Magnitude (Entire image/detail). These were expected to (...) modulate the quantity of material and textural information afforded by the image. While the Detail condition afforded less content-related information, our results show that it augmented the image’s haptic appeal. High Resolution improved the haptic and aesthetic properties of the images. Furthermore, aesthetic ratings positively correlated with sensorimotor ratings. Our results demonstrate a strict relation between the aesthetic and sensorimotor/haptic qualities of the images, empirically establishing a relationship between beholders’ bodily involvement and their aesthetic judgment of visual works of art. In addition, we found that beholders’ oculomotor behavior is selectively modulated by the perceptual manipulations being performed. The eye-tracking results indicate that the observation of the Entire, original images is the only condition in which the latency of the first fixation is shorter when participants gaze to the left side of the images. These results thus demonstrate the existence of a left-side bias during the observation of digital works of art, in particular, while participants are observing their original version. (shrink)

When people are asked to estimate the probabilities of uncertain events, they often neglect the additivity principle, which requires that the probabilities assigned to an exhaustive set of outcomes should add up to 100%. Previous studies indicate that additivity neglect is dependent on response format, self-generated probability estimates being more coherent than estimates on rating scales. The present study made use of eye-tracking methodology, recording the movement, frequency and duration of fixations during the solution of ten additivity problems and (...) two control tasks. Participants produced more non-additive estimates in the Scale format than in the Self-generated format. Self-generated estimates also led to longer decision time and a higher number of repeated inspections, suggesting a deliberate comparison process. In contrast, the Scale format seemed to encourage a case-based approach where each outcome is evaluated in isolation. (shrink)

Eye-tracking technology has become a powerful tool for biomedical-related applications due to its simplicity of operation and low requirements on patient language skills. This study aims to use the machine-learning models and deep-learning networks to identify key features of eye movements in Alzheimer's Disease (AD) under specific visual tasks, thereby facilitating computer-aided diagnosis of AD. Firstly, a three-dimensional (3D) visuospatial memory task is designed to provide participants with visual stimuli while their eye-movement data are recorded and used to build (...) an eye-tracking dataset. Then, we propose a novel deep-learning-based model for identifying patients with Alzheimer's Disease (PwAD) and healthy controls (HCs) based on the collected eye-movement data. The proposed model utilizes a nested autoencoder network to extract the eye-movement features from the generated fixation heatmaps and a weight adaptive network layer for the feature fusion, which can preserve as much useful information as possible for the final binary classification. To fully verify the performance of the proposed model, we also design two types of models based on traditional machine-learning and typical deep-learning for comparison. Furthermore, we have also done ablation experiments to verify the effectiveness of each module of the proposed network. Finally, these models are evaluated by four-fold cross-validation on the built eye-tracking dataset. The proposed model shows 85% average accuracy in AD recognition, outperforming machine-learning methods and other typical deep-learning networks. (shrink)

This study investigated the perception of Mandarin tonal alternations in disyllabic words. In Mandarin, a low-dipping Tone3 is converted to a high-rising Tone2 when followed by another Tone3, known as third tone sandhi. Although previous studies showed statistically significant differences in F0 between a high-rising Sandhi-Tone3 and a Tone2, native Mandarin listeners failed to correctly categorize these two tones in perception tasks. The current study utilized the visual-world paradigm in eye-tracking to further examine whether acoustic details in lexical tone (...) aid lexical access in Mandarin. Results showed that Mandarin listeners tend to process Tone2 as Tone2 whereas they tend to first process Sandhi-T3 as both Tone3 and Tone2, then later detect the acoustic differences between the two tones revealed by the sandhi context, and finally activate the target word during lexical access. The eye-tracking results suggest that subtle acoustic details of F0 may facilitate lexical access in automatic fashion in a tone language. (shrink)

Reading is one of the most common everyday activities, yet research elucidating how affective influence reading processes and outcomes is sparse with inconsistent results. To investigate this question, we randomly assigned participants (N = 136) to happiness (positive affect), sadness (negative affect), and neutral video-induction conditions prior to engaging in self-paced reading of a long, complex science text. Participants completed assessments targeting multiple levels of comprehension (e.g. recognising factual information, integrating different textual components, and open-ended responses of concepts from memory) (...) after reading and after a week-long delay. Results indicated that the Sadness (vs. Happiness) condition had higher comprehension scores, with the largest effects emerging for assessments targeting deeper levels comprehension immediately after reading. Eye-tracking analyses revealed that such benefits may be partly driven by sustained attentional focus over the 20-minute reading session. We discuss results with respect to theories on affect, cognition, and text comprehension. (shrink)

Two studies investigated the interaction between utterance and scene processing by monitoring eye movements in agent–action–patient events, while participants listened to related utterances. The aim of Experiment 1 was to determine if and when depicted events are used for thematic role assignment and structural disambiguation of temporarily ambiguous English sentences. Shortly after the verb identified relevant depicted actions, eye movements in the event scenes revealed disambiguation. Experiment 2 investigated the relative importance of linguistic/world knowledge and scene information. When the verb (...) identified either only the stereotypical agent of a (nondepicted) action, or the (nonstereotypical) agent of a depicted action as relevant, verb‐based thematic knowledge and depicted action each rapidly influenced comprehension. In contrast, when the verb identified both of these agents as relevant, the gaze pattern suggested a preferred reliance of comprehension on depicted events over stereotypical thematic knowledge for thematic interpretation. We relate our findings to language comprehension and acquisition theories. (shrink)

Cognitive architectures have often been applied to data from individual experiments. In this paper, I develop an ACT-R reader that can model a much larger set of data, eye-tracking corpus data. It is shown that the resulting model has a good fit to the data for the considered low-level processes. Unlike previous related works, the model achieves the fit by estimating free parameters of ACT-R using Bayesian estimation and Markov-Chain Monte Carlo techniques, rather than by relying on the mix (...) of manual selection + default values. The method used in the paper is generalizable beyond this particular model and data set and could be used on other ACT-R models. (shrink)

Crying is an ubiquitous human behavior through which an emotion is expressed on the face together with visible tears and constitutes a slippery riddle for researchers. To provide an answer to the question “How our gaze reacts to another person’s tears?,” we made use of eye tracking technology to study a series of visual stimuli. By presenting an illustrative example through an experimental setting specifically designed to study the “tearing effect,” the present work aims to offer methodological insight on (...) how to use eye-tracking technology to study non-verbal cues. A sample of 30 healthy young women with normal visual acuity performed a within-subjects task in which they evaluated images of real faces with and without tears while their eye movements were tracked. Tears were found to be a magnet for visual attention in the task of facial attribution, facilitating a greater perception of emotional intensity. Moreover, the inspection pattern changed qualitatively and quantitatively, with our participants becoming fully focused on the tears when they were visible. The mere presence of a single tear running down a cheek was associated with an increased emotional inference and greater perception of sincerity. Using normalized and validated tools (Reading the Eyes in the Mind Test and the SALAMANCA screening test for personality disorders), we measured the influence of certain characteristics of the participants on their performance of the experimental task. On the one hand, a higher level of cognitive empathy helped to classify tearful faces with higher emotional intensity and tearless faces with less emotional intensity. On the other hand, we observed that less sincerity was attributed to the tearful faces as the SALAMANCA test scores rose in clusters A (strange and extravagant) and B (immature and emotionally unstable) of our sample. The present findings highlight the advantages of using eye tracking technology to study non-verbal cues and draw attention to methodological issues that should be taken into account. Further exploration of the relationship between empathy and tear perception could be a fruitful avenue of future research using eye tracking. (shrink)

Starting with the hypothesis that analogical reasoning consists of a search of semantic space, we used eye-tracking to study the time course of information integration in adults in various formats of analogies. The two main questions we asked were whether adults would follow the same search strategies for different types of analogical problems and levels of complexity and how they would adapt their search to the difficulty of the task. We compared these results to predictions from the literature. Machine (...) learning techniques, in particular support vector machines (SVMs), processed the data to find out which sets of transitions best predicted the output of a trial (error or correct) or the type of analogy (simple or complex). Results revealed common search patterns, but with local adaptations to the specifics of each type of problem, both in terms of looking-time durations and the number and types of saccades. In general, participants organized their search around source-domain relations that they generalized to the target domain. However, somewhat surprisingly, over the course of the entire trial, their search included, not only semantically related distractors, but also unrelated distractors, depending on the difficulty of the trial. An SVM analysis revealed which types of transitions are able to discriminate between analogy tasks. We discuss these results in light of existing models of analogical reasoning. (shrink)

This study investigated the impact of online product reviews on consumers purchasing decisions by using eye-tracking. The research methodology involved development of a conceptual framework of online product review and purchasing intention through the moderation role of gender and visual attention in comments, and empirical investigation into the region of interest analysis of consumers fixation during the purchase decision process and behavioral analysis. The results showed that consumers’ attention to negative comments was significantly greater than that to positive comments, (...) especially for female consumers. Furthermore, the study identified a significant correlation between the visual browsing behavior of consumers and their purchase intention. It also found that consumers were not able to identify false comments. The current study provides a deep understanding of the underlying mechanism of how online reviews influence shopping behavior, reveals the effect of gender on this effect for the first time and explains it from the perspective of attentional bias, which is essential for the theory of online consumer behavior. Specifically, the different effects of consumers’ attention to negative comments seem to be moderated through gender with female consumers’ attention to negative comments being significantly greater than to positive ones. These findings suggest that practitioners need to pay particular attention to negative comments and resolve them promptly through the customization of product/service information, taking into consideration consumer characteristics, including gender. (shrink)

Cue labels are useful during multimedia learning. According to spatial contiguity principle, people learn more when related words and pictures are displayed spatially near one another. Well-arranged labels of multimedia material can greatly facilitate learning. This study used eye tracking to examine the joint influence of label size and color on multimedia learning. The results revealed that larger labels led to better retention test performance and a higher AOI glance count, but no cueing effect was found for color. Cues (...) have a certain attention-leading function that promotes the learner remembering the content. These findings suggest that salient labels that provide explanatory information can guide learners’ attention and facilitate learning, though a combination of label size and color salience did not demonstrate a superior cueing effect. (shrink)

Background: Facial attractiveness influences our perceptions of others, with beautiful faces reaping societal rewards and anomalous faces encountering penalties. The purpose of this study was to determine associations of visual attention with bias and social dispositions toward people with facial anomalies. -/- Methods: Sixty subjects completed tests evaluating implicit bias, explicit bias, and social dispositions before viewing publicly available images of preoperative and postoperative patients with hemifacial microsomia. Eye-tracking was used to register visual fixations. -/- Results: Participants with higher (...) implicit bias scores fixated significantly less on the cheek and ear region preoperatively (P = 0.004). Participants with higher scores in empathic concern and perspective taking fixated more on the forehead and orbit preoperatively (P = 0.045) and nose and lips (P = 0.027) preoperativel. -/- Conclusions: Participants with higher levels of implicit bias spent less visual attention on anomalous facial anatomy, whereas participants with higher levels of empathic concern and perspective taking spent more visual attention on normal facial anatomy. Levels of bias and social dispositions such as empathy may predict layperson gaze patterns toward those with facial anomalies and provide insights to neural mechanisms underlying the "anomalous is bad" paradigm. (shrink)

This study looked for evidence of biases in the allocation and disengagement of attention in dysphoric individuals. Participants studied images for a recognition memory test while their eye fixations were tracked and recorded. Four image types were presented (depression-related, anxiety-related, positive, neutral) in each of two study conditions. For the simultaneous study condition, four images (one of each type) were presented simultaneously for 10 seconds, and the number of fixations and the total fixation time to each image was measured, similar (...) to the procedure used by Eizenman et al. (2003) and Kellough, Beevers, Ellis, and Wells (2008). For the sequential study condition, four images (one of each type) were presented consecutively, each for 4 seconds; to measure disengagement of attention an endogenous cuing procedure was used (Posner, 1980). Dysphoric individuals spent significantly less time attending to positive images than non-dysphoric individuals, but there were no group differences in attention to depression-related images. There was also no evidence of a dysphoria-related bias in initial shifts of attention. With respect to the disengagement of attention, dysphoric individuals were slower to disengage their attention from depression-related images. The recognition memory data showed that dysphoric individuals had poorer memory for emotional images, but there was no evidence of a conventional mood-congruent memory bias. Differences in the attentional and memory biases observed in depressed and dysphoric individuals are discussed. (shrink)

Background: Traditional residential architecture often integrates intricate decorative patterns, vital in shaping the overall aesthetic experience. Aim: The purpose of this research is to investigate the visual perception of traditional residential architectural decorations using eye-tracking experiments, focusing on various design elements influence aesthetic appreciation. Methodology: In this study, we used eye-tracking equipment to record participants' gaze patterns and fixation lengths while they viewed images of traditional architectural decorations. 95 participants from diverse demographic backgrounds in the Leizhou Peninsula were (...) involved in the experiments. The eye tracking data was analysed to determine how participants' visual attention varied based on the attractiveness and complexity of the decorative patterns. We examined eye movements to evaluate fixation counts, duration, and pupil dilation using electroencephalogram (EEG). Additionally, subjective feedback was collected through surveys to gauge participants' aesthetic preferences. Result and Conclusion: Findings indicate that more elaborate and visually appealing traditional decorations elicited longer fixation durations and higher levels of engagement compared to simpler designs. This study highlights the significant impact of traditional decorative patterns on visual perception and aesthetic appreciation. These findings can guide the design of modern residential products, ensuring they resonate with cultural aesthetics while enhancing the user experience. (shrink)

Installation art is one of the most important and provocative developments in the visual arts during the last half century and has become a key focus of artists and of contemporary museums. It is also seen as particularly challenging or even disliked by many viewers, and—due to its unique in situ, immersive setting—is equally regarded as difficult or even beyond the grasp of present methods in empirical aesthetic psychology. In this paper, we introduce an exploratory study with installation art, utilizing (...) a collection of techniques to capture the eclectic, the embodied, and often the emotionally-charged viewing experience. We present results from an investigation of two pieces, both part of Olafur Eliasson’s exhibition “Baroque, Baroque” held at the Belvedere museum in Vienna. These were assessed by pre- and post-viewing questionnaires focusing on emotion, meaning-making, and appraisals, in tandem with mobile eye tracking to consider viewers’ attention to both installed artworks and/or to the museum environment. The data showed differences in participants’ emotional states, appraisals, and visual exploration, which together paint a picture of the aesthetic reactions to the works. These differences also showed how viewers’ appraisal strategies, meaning making, and physical actions facilitated relatively more or less deep engagement with, and enjoyment of, the art. The results are discussed in terms of their implications for museum studies, art education, and theory in empirical aesthetics. (shrink)

In this study, we investigate the processing of object-denoting emoji in sentences using eye tracking. We hypothesize that (a) such emoji are more difficult to process when used as word replacement; and (b) their processing is subject to ambiguity constraints similarly to what happens with words. We conduct two experiments in which participants have to read sentences in which an emoji either follows or replaces a word. Control stimuli not containing emoji are also tested. In the second experiment, the (...) emoji are presented in two different disambiguating contexts. First fixation duration, total visit duration, and total revisit duration are modeled in the various conditions using linear mixed models. Both our hypotheses are supported. We observe longer total visit time for non-redundant emoji, and higher values for all three measures for ambiguous emoji. We conclude that lexical access may be more difficult for emoji especially when they are used as word replacements and their meaning is not immediately clear. Furthermore, we also conclude that non-redundant emoji are more difficult to integrate in the processing of the sentence than the equivalent words, or emoji used in a redundant way. In turn, our results indicate that emoji may not always be as immediate and easy to process for readers in spite of their popularity. (shrink)

Anxiety can impair the central executive functioning in working memory (WM). Further, the adverse effect of anxiety on the central executive would be greater when threat-related distractors are present. This study investigated the effect of task-irrelevant emotional faces on WM updating in social anxiety. Forty-one socially anxious (SA) and thirty-nine non-anxious (NA) participants completed an emotional face interference n-back task coupled with eye movement recording. The results showed that, in the 2-back task, SA participants had longer reaction times in the (...) angry-neutral and neutral-neutral interference conditions than in the no-interference condition, whereas NA participants had longer reaction times in the happy-neutral and neutral-neutral interference conditions than in the no-interference condition. In addition, SA participants initially fixated on angry faces more frequently and spent more time looking at them, whereas NA participants initially fixated on happy faces more frequently and spent more time looking at them. This study suggests that attentional bias towards social threats reduces the efficiency rather than effectiveness of WM updating in social anxiety. Moreover, SA individuals are better at resisting interference from task-irrelevant positive stimuli, while NA individuals are better at resisting interference from task-irrelevant threatening stimuli. (shrink)

Immersive 360-degree video has become a new learning resource because of its immersive sensory experience. This study examined the effects of textual and visual cues on learning and attention in immersive 360-degree video by using eye-tracking equipment integrated in a virtual reality head-mounted display. Participants were randomly assigned to one of four conditions: no cues, textual cues in the initial field of view, textual cues outside the initial FOV, and textual cues outside the initial FOV + visual cues. The (...) results showed that the cues helped learners achieve better learning outcomes and spend more time focusing on the areas with cues. In addition, the study found a serious imbalance in the distribution of learners’ attention in each region of the video. The attention directed to textual cues in the initial FOV is much higher than the attention directed to textual cues outside the initial FOV. Adding visual cues can effectively direct attention to textual cues outside the initial FOV and alleviate the imbalance of attention distribution. Consequently, adding cues to immersive 360-degree video can be an appropriate approach to promote learning and guide attention in immersive 360-degree video learning environments. This study provided new insights into the design and development of immersive 360-degree video instructional resources. (shrink)