This chapter argues that emotion recognition is a skill. A skill perspective on emotion recognition draws attention to underappreciated features of this cornerstone of social cognition. Skills have a number of characteristic features. For example, they are improvable, practical, and flexible. Emotion recognition has these features as well. Leading theories of emotion recognition often draw inadequate attention to these features. The chapter advances a theory of emotion recognition that is better (...) suited to this purpose. It proposes that emotion recognition involves scripts. Emotion scripts describe how people are likely to behave in different emotional contexts. Scripts can be improved with the addition of richer social knowledge, they are practical in that they describe and guide social interaction, and they are flexible because they must accommodate the fact that emotions have different behavioral impact depending on the agents involved and the circumstances at play. Learning to recognize emotions through scripts qualifies as a skill. (shrink)

There is debate within the literature as to whether emotion dysregulation (ED) in Attention-Deficit Hyperactivity Disorder (ADHD) reflects deviant attentional mechanisms or atypical perceptual emotion processing. Previous reviews have reliably examined the nature of facial, but not vocal, emotion recognition accuracy in ADHD. The present meta-analysis quantified vocal emotion recognition (VER) accuracy scores in ADHD and controls using robust variance estimation, gathered from 21 published and unpublished papers. Additional moderator analyses were carried out to (...) determine whether the nature of VER accuracy in ADHD varied depending on emotion type. Findings revealed a medium effect size for the presence of VER deficits in ADHD, and moderator analyses showed VER accuracy in ADHD did not differ due to emotion type. These results support the theories which implicate the role of attentional mechanisms in driving VER deficits in ADHD. However, there is insufficient data within the behavioural VER literature to support the presence of emotion processing atypicalities in ADHD. Future neuro-imaging research could explore the interaction between attention and emotion processing in ADHD, taking into consideration ADHD subtypes and comorbidities. (shrink)

We develop a version of a direct perception account of emotion recognition on the basis of a metaphysical claim that emotions are individuated as patterns of characteristic features. On our account, emotion recognition relies on the same type of pattern recognition as is described for object recognition. The analogy allows us to distinguish two forms of directly perceiving emotions, namely perceiving an emotion in the absence of any top-down processes, and perceiving an (...) class='Hi'>emotion in a way that significantly involves some top-down processes ; and, in addition, an inference-based evaluation of an emotion. Our model clarifies the epistemology of emotion recognition. (shrink)

According to a dominant interpretation of the simulation hypothesis, in recognizing an emotion we use the same neural processes used in experiencing that emotion. This paper argues that the view is fundamentally misguided. I will examine the simulational arguments for the three basic emotions of fear, disgust, and anger and argue that the simulational account relies strongly on a narrow sense of emotion processing which hardly squares with evidence on how, in fact, emotion recognition is (...) processed. I contend that the current body of empirical evidence suggests that emotion recognition is processed in an integrative system involving multiple cross-regional interactions in the brain, a view which squares with understanding emotion recognition as an information-rich, rather than simulational, process. In the final section, I discuss possible objections. (shrink)

BackgroundWe review original papers on ovarian-hormone status in two areas of emotional processing: facial emotion recognition and emotional memory. Ovarian-hormone status is operationalized by the levels of the steroid sex hormones 17β-estradiol and progesterone, fluctuating over the natural menstrual cycle and suppressed under oral contraceptive use. We extend previous reviews addressing single areas of emotional processing. Moreover, we systematically examine the role of stimulus features such as emotion type or stimulus valence and aim at elucidating factors that (...) reconcile the inconsistent results.MethodsWe followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and included papers published until September 2020 indexed in PubMed and Web of Science databases. Search terms were MeSH terms AND AND with representing our separately searched areas, resulting in, and. To be included, articles had to be written and published in English, examine healthy, non-pregnant adult women in their reproductive age, and measure or at least estimate levels of E2 and P4. In PubMed, the search was limited to humans and to the search term present in the title or abstract.ResultsFeatures of the provided stimulus material constitute a relevant influence that interacts with E2- and P4-related ovarian-hormone status. For instance, recognition of basic emotions appears to be more related to P4- than E2-levels. Quite consistent, OC intake was accompanied by impaired recognition accuracy of basic and also complex emotions, although not in a recent large-sample study assessing complex emotions. Memory recall of negative content was mainly enhanced by P4, especially after having been stressed.Discussion and ConclusionWe document the methodological diversity in the field, presumably contributing to the heterogeneity of results. More studies explicitly contrasting the early follicular phase, mid-cycle phase, mid-luteal, and OC intake while standardizing tasks are needed. Research would take advantage of using within-subject designs and accounting for the recognition of complex emotions. (shrink)

BackgroundParanoia is associated with a multitude of social cognitive deficits, observed in both clinical and subclinical populations. Empathy is significantly and broadly impaired in schizophrenia, yet its relationship with subclinical paranoia is poorly understood. Furthermore, deficits in emotion recognition – a very early component of empathic processing – are present in both clinical and subclinical paranoia. Deficits in emotion recognition may therefore underlie relationships between paranoia and empathic processing. The current investigation aims to add to the (...) literature on social cognition and paranoia by: (1) characterizing the relationship between paranoia and empathy, and (2) testing whether there is an indirect effect of emotion recognition on the relationship between empathy and paranoia.MethodsParanoia, empathy, and emotion recognition were assessed in a non-clinical sample of adults (n= 226) from the Nathan Kline Institute-Rockland (NKI-Rockland) dataset. Paranoia was measured using the Peters Delusions Inventory-21 (PDI-21). Empathy was measured using the Interpersonal Reactivity Index (IRI), a self-report instrument designed to assess empathy using four subscales: Personal Distress, Empathic Concern, Perspective Taking, and Fantasy. Emotion recognition was assessed using the Penn Emotion Recognition Test (ER-40). Structural equation modeling (SEM) was used to estimate relationships between paranoia, the four measures of empathy and emotion recognition.ResultsParanoia was associated with the Fantasy subscale of the IRI, such that higher Fantasy was associated with more severe paranoia (p< 0.001). No other empathy subscales were associated with paranoia. Fantasy was also associated with the emotion recognition of fear, such that higher Fantasy was correlated with better recognition of fear (p= 0.008). Paranoia and emotion recognition were not significantly associated. The Empathic Concern subscale was negatively associated with emotion recognition, with higher empathic concern related to worse overall emotion recognition (p= 0.002). All indirect paths through emotion recognition were non-significant.DiscussionThese results suggest that imaginative perspective-taking contributes to paranoia in the general population. These data do not, however, point to robust global relationships between empathy and paranoia or to emotion recognition as an underlying mechanism. Deficits in empathy and emotion recognition observed in schizophrenia may be associated with the broader pathology of schizophrenia, and therefore not detectable with subclinical populations. (shrink)

Despite the lower levels of psychopathy in women than in men, the scientific interest in studying psychopathy in female participants is increasing. Nevertheless, the number of studies investigating psychopathy in women and associated phenomena remains low. The influence of psychopathy in women inmates on experimental tasks of emotional recognition and moral judgment was evaluated, aiming to contribute to this field of research. Utilitarian moral judgment was predicted by psychopathy, specifically by primary and secondary psychopathy, while primary psychopathy predicted a (...) worse performance on the emotional recognition task. There was no significant influence of general intellectual abilities on either task. Contrarily to what was expected, emotional recognition did not prove to be a significant mediator of the relationship between psychopathy and utilitarian moral judgment. These results emphasize that the tendency to utilitarian moral judgment and worse recognition of facial expressions of emotion are associated to higher psychopathy scores in female inmates (especially primary psychopathy), but more studies are necessary to address the role of the emotional component in the process of moral judgment. (shrink)

Emotion recognition technology uses artificial intelligence to make inferences about a person’s emotions, on the basis of their facial expressions, body language, tone of voice, or other types of input. Underlying such technology are a variety of assumptions about the manifestation, nature, and value of emotions. To assure the quality and desirability of emotion recognition technology, it is important to critically assess the assumptions embedded in the technology. Within philosophy, there is a long tradition of epistemological, (...) ontological, phenomenological, and ethical reflection on the manifestation, nature, and value of emotions. This article draws from this tradition of philosophy of emotions, in order to challenge the assumptions underlying current emotion recognition technology and to promote a more critical engagement with the concept of emotions in the tech-industry. (shrink)

The artificial neural network is employed to study children’s psychological emotion recognition to fully reflect the psychological status of preschool children and promote the healthy growth of preschool children. Specifically, the ANN model is used to construct the human physiological signal measurement platform and emotion recognition platform to measure the human physiological signals in different psychological and emotional states. Finally, the parameter values are analyzed on the emotion recognition platform to identify the children’s psychological (...) and emotional states accurately. The experimental results demonstrate that the recognition ability of children aged 4–6 to recognize the three basic emotions of happiness, calm, and fear increases with age. Besides, there are significant age differences in children’s recognition of happiness, calm, and fear. In addition, the effect of 4-year-old children on the theory of mind tasks is less than that of 5- to 6-year-old children, which may be related to more complex cognitive processes. Preschool children are experiencing a stage of rapid emotional development. If children cannot be guided to reasonably identify and deal with emotions at this stage, their education level and social ability development will be significantly affected. Therefore, this study has significant reference value for preschool children’s emotional recognition and guidance and can promote children’s emotional processing and mental health. (shrink)

There exists substantial cultural variation in how emotions are expressed, but there is also considerable evidence for universal properties in facial and vocal affective expressions. This is the first empirical effort examining the perception of vocal emotional expressions across cultures with little common exposure to sources of emotion stimuli, such as mass media. Shuar hunter-horticulturalists from Amazonian Ecuador were able to reliably identify happy, angry, fearful and sad vocalizations produced by American native English speakers by matching emotional spoken utterances (...) to emotional expressions portrayed in pictured faces. The Shuar performed similarly to English speakers who heard the same utterances in a content-filtered condition. These data support the hypothesis that vocal emotional expressions of basic affective categories manifest themselves in similar ways across quite disparate cultures. (shrink)

At present, there are so many learners in online classroom that teachers cannot master the learning situation of each student comprehensively and in real time. Therefore, this paper first constructs a multimodal emotion recognition model based on CNN-BiGRU. Through the feature extraction of video and voice information, combined with temporal attention mechanism, the attention distribution of each modal information at different times is calculated in real time. In addition, based on the recognition of learners’ emotions, a prediction (...) model of learners’ achievement based on emotional state assessment is proposed. C4.5 algorithm is used to predict students’ academic achievement in the multi-polarized emotional state, and the relationship between confusion and academic achievement is further explored. The experimental results show that the proposed multi-scale self-attention layer and multi-modal fusion layer can improve the achievement of ER task; moreover, there is a strong correlation between students’ confusion and foreign language achievement. Finally, the model can accurately and continuously observe students’ learning emotion and state, which provides a new idea for the reform of education modernization. (shrink)

Linguistic communication is an important part of the cross-cultural perspective, and linguistic textual emotion recognition is a key massage in interpersonal communication. Spanish is the second largest language system in the world. The purpose of this paper is to identify the emotional features in Spanish texts. The improved BiLSTM framework is proposed. We select three widely used Spanish dictionaries as the datasets for our experiments, and then we finally obtain text sentiment classification results through text preprocessing, text (...) class='Hi'>emotion feature extraction, text topic detection, and emotion classification. We inserted the attention mechanism in the improved BiLSTM framework. It enables the shared feature encoder to obtain weighted representation results in the extraction of emotion features, which enhances the generalization ability of the model for text emotion feature recognition. Experimental results demonstrate that our approach performs better for specialized Spanish dictionary datasets. In terms of emotion recognition accuracy, the average value is as high as 76.21%. The overall performance outperforms current comparable machine learning methods and convolutional neural network methods. (shrink)

Financial market and economic growth and development trends can be regarded as an extremely complex system, and the in-depth study and prediction of this complex system has always been the focus of attention of economists and other scholars. Emotion recognition algorithm is a pattern recognition technology that integrates a number of emerging science and technology, and has good non-linear system fitting capabilities. However, using emotion recognition algorithm models to analyze and predict financial market and economic (...) growth and development trends can yield more accurate prediction results. This article first gives a detailed introduction to the existing financial development and economic growth status and development trend forecasting problems, and then gives a brief overview of the concept of emotion recognition algorithms. Then, it describes the emotion recognition methods, including statistical emotion recognition methods, mixed emotion recognition methods, and emotion recognition methods based on knowledge technology, and conducts in-depth research on the three algorithm models of statistical emotion recognition methods, they are the support vector machine algorithm model, the artificial neural network algorithm model, and the long and short-term memory network algorithm model. Finally, these three algorithm models are applied to the financial market and economic growth and development trend prediction experiments. Experimental results show that the average absolute error of the three algorithms is below 25, which verifies that the emotion recognition algorithm has good operability and feasibility for the prediction of financial market and economic growth and development trends. (shrink)

Emotion recognition skills and the ability to understand the mental states of others are crucial for normal social functioning. Conversely, delays and impairments in these processes can have a profound impact on capability to engage in, maintain, and effectively regulate social interactions. Therefore, this study aimed to compare the performance of 42 autistic children, 45 unaffected siblings, and 41 typically developing controls on the Affect Recognition and Theory of Mind subtests of the Developmental Neuropsychological Assessment Battery. There (...) were no significant differences between siblings and TD controls. Autistic children showed significantly poorer performance on AR when compared to TD controls and on TOM when compared to both TD controls and unaffected siblings. An additional comparison of ASD, unaffected sibling and TD control subsamples, matched on full-scale IQ, revealed no group differences for either AR or TOM. AR and TOM processes have received less research attention in siblings of autistic children and remain less well characterized. Therefore, despite limitations, findings reported here contribute to our growing understanding of AR and TOM abilities in siblings of autistic children and highlight important future research directions. (shrink)

Purpose Uncertainty is an under-respected issue when it comes to automatic assessment of human emotion by machines. The purpose of this paper is to highlight the existent approaches towards such measurement of uncertainty, and identify further research need. Design/methodology/approach The discussion is based on a literature review. Findings Technical solutions towards measurement of uncertainty in automatic emotion recognition exist but need to be extended to respect a range of so far underrepresented sources of uncertainty. These then need (...) to be integrated into systems available to general users. Research limitations/implications Not all sources of uncertainty in automatic emotion recognition including emotion representation and annotation can be touched upon in this communication. Practical implications AER systems shall be enhanced by more meaningful and complete information provision on the uncertainty underlying their estimates. Limitations of their applicability should be communicated to users. Social implications Users of automatic emotion recognition technology will become aware of their limitations, potentially leading to a fairer usage in crucial application context. Originality/value There is no previous discussion including the technical view point on extended uncertainty measurement in automatic emotion recognition. (shrink)

Background: Clinical and experimental findings have suggested that insomnia is associated with altered emotion processing, such as facial emotion recognition and impairments in executive functions. However, the results still appear non-consensual and have recently been presented by a few number of studies. Accordingly, the aim of the present study was to investigate whether patients with Insomnia disorder will present alterations in recognition of facial emotions and that such alterations will be related to Executive Functions and that (...) Insomnia Disorder patients will exhibit a lower Executive Functions performance when compared to normal controls individuals. Method: A total of 11 individuals diagnosed with insomnia disorder and 15 control participants took part in the present cross-sectional study. They completed questionnaires covering socio-demographic data, a sleep and mood evaluation, and a cognitive assessment. Moreover, they underwent a computerized Facial Emotional Recognition test, consisting of facial emotion labeling and facial emotion matching. Results: Individuals with insomnia presented a lower recognition of facial expressions of fear and sadness in comparison to control volunteers. Perceptual Organization Index was found to be statistically significant lower in the insomnia group compared to the control group. There was a statistically significant association between facial recognition emotional and cognitive processes of in executive functions performance, especially in processes related to the inhibitory control, planning capacity, problem solving and cognitive flexibility in insomnia group. Conclusions: Insomnia disorder impairs the facial emotion recognition of the fear and sadness. Insomniacs with impairment in the facial emotion recognition had impaired performance of executive functions. There is a negative impact on processes related to cognitive-emotional capacity than on processes related to social-emotional recognition. Keywords: insomnia, facial emotional recognition, sleep deprivation, cognition, executive functions. (shrink)

Most research on cross-cultural emotion recognition has focused on facial expressions. To integrate the body of evidence on vocal expression, we present a meta-analysis of 37 cross-cultural studies...

Eliciting negative stereotypes about ageing commonly results in worse performance on many physical, memory, and cognitive tasks in adults aged over 65. The current studies explored the potential effect of this “stereotype threat” phenomenon on older adults’ emotion recognition, a cognitive ability that has been demonstrated to decline with age. In Study 1, stereotypes about emotion recognition ability across the lifespan were established. In Study 2, these stereotypes were utilised in a stereotype threat manipulation that framed (...) an emotion recognition task as assessing either cognitive ability, social ability, or general abilities. Participants then completed an emotion recognition task in which they labelled dynamic expressions of negative and positive emotions. Self-reported threat concerns were also measured. Framing an emotion recognition task as assessing cognitive ability significantly heightened older adults’ reports of stereotype threat concerns. Despite this, older adults’ emotion recognition performance was unaffected. Unlike other cognitive abilities, recognising facially expressed emotions may be unaffected by stereotype threat, possibly because emotion recognition is automatic, making it less susceptible to the cognitive load that stereotype threat produces. (shrink)

The purpose of this study was to investigate emotion recognition and processing in patients with mild cognitive impairment in order to update the state of current literature on this important but undervalued topic. We identified 15 papers published between 2012 and 2022 that meet the inclusion criteria. Paper search, selection, and extraction followed the PRISMA guidelines. We used a narrative synthesis approach in order to report a summary of the main findings taken from all papers. The results collected (...) are still ambiguous: some studies did not find any differences between MCI and healthy controls groups in emotion recognition and processing, and other results reported emotion-specific deficits in emotion recognition regarding MCI patients. It is essential to underline that these findings could not be generalized to the whole MCI population due to the heterogeneous use of measures and composition of the sample. This does not allow us to make a comprehensive comparison between the results. Our suggestion for future research is to align the results using the same type of tests and emotion recognition assessment. (shrink)

As an important component to promote the development of affective brain–computer interfaces, the study of emotion recognition based on electroencephalography (EEG) has encountered a difficult challenge; the distribution of EEG data changes among different subjects and at different time periods. Domain adaptation methods can effectively alleviate the generalization problem of EEG emotion recognition models. However, most of them treat multiple source domains, with significantly different distributions, as one single source domain, and only adapt the cross-domain marginal (...) distribution while ignoring the joint distribution difference between the domains. To gain the advantages of multiple source distributions, and better match the distributions of the source and target domains, this paper proposes a novel multi-source joint domain adaptation (MSJDA) network. We first map all domains to a shared feature space and then align the joint distributions of the further extracted private representations and the corresponding classification predictions for each pair of source and target domains. Extensive cross-subject and cross-session experiments on the benchmark dataset, SEED, demonstrate the effectiveness of the proposed model, where more significant classification results are obtained on the more difficult cross-subject emotion recognition task. (shrink)

ABSTRACTRecently, Cognition and Emotion published an article demonstrating that age cues affect the speed and accuracy of emotion recognition. The authors claimed that the observed effect of target age on emotion recognition is better explained by evaluative than stereotype associations. Although we agree with their conclusion, we believe that with the research method the authors employed, it was impossible to detect a stereotype effect to begin with. In the current research, we successfully replicate previous findings. (...) Furthermore, by changing the comparative context, Study 2 provides a first test of age-stereotypes affecting emotions recognition. We discuss recommendations for future studies in the domain of social categorisation and emotion recognition. (shrink)

Purpose This study responds to Agnieszka Landowska’s paper about the lack of accuracy in emotion recognition. Design/methodology/approach The approach is purely theoretical. The paper also refers to empirical studies. Findings The author first elaborates on Landowska’s “postulates” and then shortly expands on how virtual chatbots such as “AI therapists” pose considerable challenges to emotion recognition algorithms as well. Originality/value This viewpoint’s value is to elaborate and expand on an ongoing discussion on emotion recognition technologies.

One of the significant courses in Chinese universities is English. This course is usually taught by a foreign language instructor. There will, however, necessarily be some communication hurdles between “foreign language teachers” and “native students.” This research presents an emotion recognition method for foreign language teachers in order to eliminate communication barriers between teachers and students and improve student learning efficiency. We discovered four factors of emotion recognition through literature analysis: smile, eye contact, gesture, and tone. (...) We believe that differences in foreign language teachers’ performance in these four areas will have an impact on students’ emotion recognition and, as a result, on their learning efficiency. The influence of the foreign language teacher’s eye contact and gestures is larger in the decision whether can improve the students’ classroom learning efficiency, according to 43 of the questionnaire data analysis. The second is the tone and smile. Our research contributes to the body of knowledge on emotion recognition in university foreign language teachers by presenting a practical method for recognizing emotion in foreign language teachers. We recommend that college foreign language teachers pay attention to eye and gesture communication with students in English classrooms based on the findings. By enriching the style of emotional expression in class, college language teachers, particularly foreign language teachers, can improve communication, and connection with students. (shrink)

Facial emotion recognition (FER) deficits interfere with interpretation of social situations and selection of appropriate responses. Attention-Deficit/Hyperactivity Disorder (ADHD) symptoms are independently associated with social difficulties and might exacerbate the influence of deficient FER, because children with ADHD symptoms have fewer compensatory resources in social situations when they misinterpret emotions. Very few studies have tested this hypothesis in a community context, where child ADHD symptoms vary on a continuum. The current study extended this work by utilising a community (...) sample (N = 87) of boys and girls in middle childhood (M = 7.83 years) and testing for moderation of FER effects separately by ADHD symptom type (ADHD-I = inattentive, H = hyperactive/impulsive, C = combined) using linear regression. While lower FER was associated with more social problems, this relationship was qualified by the presence of ADHD symptoms. Specifically, only children with relatively high ADHD symptoms in our community sample showed this inverse relationship, which was clearest among children with elevated ADHD-C or ADHD-I symptoms. No gender differences were observed. These results support our primary hypothesis, extend prior findings to boys and girls in the community, and have implications for understanding how ADHD symptoms and FER influence youth social deficits. (shrink)

Results from studies on gender differences in emotion recognition vary, depending on the types of emotion and the sensory modalities used for stimulus presentation. This makes comparability between different studies problematic. This study investigated emotion recognition of healthy participants (N = 84; 40 males; ages 20 to 70 years), using dynamic stimuli, displayed by two genders in three different sensory modalities (auditory, visual, audio-visual) and five emotional categories. The participants were asked to categorise the stimuli (...) on the basis of their nonverbal emotional content (happy, alluring, neutral, angry, and disgusted). Hit rates and category selection biases were analysed. Women were found to be more accurate in recognition of emotional prosody. This effect was partially mediated by hearing loss for the frequency of 8,000 Hz. Moreover, there was a gender-specific selection bias for alluring stimuli: Men, as compared to women, chose “alluring” more often when a stimulus was presented by a woman as compared to a man. (shrink)

ABSTRACTPrevious research on the development of emotion recognition in music has focused on classical, rather than popular music. Such research does not consider the impact of lyrics on judgements...

Emotion recognition is an important part of building an intelligent human-computer interaction system and plays an important role in human-computer interaction. Often, people express their feelings through a variety of symbols, such as words and facial expressions. A business ecosystem is an economic community based on interacting organizations and individuals. Over time, they develop their capabilities and roles together and tend to develop themselves in the direction of one or more central enterprises. This paper aims to study a (...) multimodal ER method based on attention mechanism. It analyzes the current emotional state of consumers and the development direction of enterprises through multi-modal ER of human emotions and analysis of market trends, so as to provide the most appropriate response or plan. This paper firstly describes the related methods of multimodal ER and deep learning in detail, and briefly outlines the meaning of enterprise strategy in the business ecosystem. Then, two datasets, CMU-MOSI and CMU-MOSEI, are selected to design the scheme for multimodal ER based on self-attention mechanism. Through the comparative analysis of the accuracy of single-modal and multi-modal ER, the self-attention mechanism is applied in the experiment. The experimental results show that the average recognition accuracy of happy under multimodal ER reaches 91.5%. (shrink)

To define the determinants of impaired facial emotion recognition (FER) in patients with mesial temporal lobe epilepsy associated with hippocampal sclerosis (MTLE-HS), we examined 76 patients with unilateral MTLE-HS, 36 prior to antero-mesial temporal lobectomy (AMTL) and 40 after AMTL, and 28 healthy control subjects with a FER test consisting of 60 items (20 each for anger, fear, and happiness). Mean percentages of the accurate responses were calculated for different subgroups: right vs. left MTLE-HS, early (age at onset (...) <6 years) vs. late-onset, and before vs. after AMTL. After controlling for years of education, duration of epilepsy and number of antiepileptic drugs (AEDs) taken, on multivariate analysis, fear recognition was profoundly impaired in early-onset right MTLE-HS patients compared to other MTLE patients and control subjects. Happiness recognition was significantly better in post-AMTL MTLE-HS patients compared to pre-AMTL patients while anger and fear recognition did not differ. We conclude that patients with right MTLE-HS with age at seizure onset <6 years are maximally predisposed to impaired fear recognition. In them, right AMTL does not further worsen FER abilities. Longitudinal studies comparing FER in the same patients before and after AMTL will be required to refine and confirm our cross-sectional observations. (shrink)

With regard to facial emotion recognition, previous studies found that specific facial regions were attended more in order to identify certain emotions. We investigated whether a preferential search for emotion-specific diagnostic regions could contribute toward the accurate recognition of facial emotions. Twenty-three neurotypical adults performed an emotion recognition task using six basic emotions: anger, disgust, fear, happiness, sadness, and surprise. The participants’ exploration patterns for the faces were measured using the Moving Window Technique (MWT). (...) This technique presented a small window on a blurred face, and the participants explored the face stimuli through a mouse-controlled window in order to recognize the emotions on the face. Our results revealed that when the participants explored the diagnostic regions for each emotion more frequently, the correct recognition of the emotions occurred at a faster rate. To the best of our knowledge, this current study is the first to present evidence that an exploration of emotion-specific diagnostic regions can predict the reaction time of accurate emotion recognition among neurotypical adults. Such findings can be further applied in the evaluation and/or training (regarding emotion recognition functions) of both typically and atypically developing children with emotion recognition difficulties. (shrink)

Emotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from our facial expressions by asking whether this task can be performed reliably. This means, rather than considering the potential harms or scientific soundness of facial emotion recognition systems, focusing on the reliability of (...) the ground truths used to develop emotion recognition systems, assessing how well different human observers agree on the emotions they detect in subjects’ faces. Additionally, we discuss the extent to which sharing context can help observers agree on the emotions they perceive on subjects’ faces. Briefly, we demonstrate that when large and heterogeneous samples of observers are involved, the task of emotion detection from static images crumbles into inconsistency. We thus reveal that any endeavour to understand human behaviour from large sets of labelled patterns is over-ambitious, even if it were technically feasible. We conclude that we cannot speak of actual accuracy for facial emotion recognition systems for any practical purposes. (shrink)

This exploration aims to study the emotion recognition of speech and graphic visualization of expressions of learners under the intelligent learning environment of the Internet. After comparing the performance of several neural network algorithms related to deep learning, an improved convolution neural network-Bi-directional Long Short-Term Memory algorithm is proposed, and a simulation experiment is conducted to verify the performance of this algorithm. The experimental results indicate that the Accuracy of CNN-BiLSTM algorithm reported here reaches 98.75%, which is at (...) least 3.15% higher than that of other algorithms. Besides, the Recall is at least 7.13% higher than that of other algorithms, and the recognition rate is not less than 90%. Evidently, the improved CNN-BiLSTM algorithm can achieve good recognition results, and provide significant experimental reference for research on learners’ emotion recognition and graphic visualization of expressions in an intelligent learning environment. (shrink)

The present study examines the association between people’s interoceptive representation of physical sensations and the recognition of vocal and facial expressions of emotion. We used body maps to study the granularity of the interoceptive conceptualisation of 11 positive emotions (amusement, awe, compassion, contentment, desire, love, joy, interest, pride, relief, and triumph) and a new emotion recognition test (Emotion Expression Understanding Test) to assess the ability to recognise emotions from vocal and facial behaviour. Overall, we found (...) evidence for distinct interoceptive conceptualizations of 11 positive emotions across Asian American, European American, and Latino/a American cultures, as well as the reliable identification of emotion in facial and vocal expressions. Central to new theorising about emotion-related representation, the granularity of physical sensations did not covary with emotion recognition accuracy, suggesting that two kinds of emotion conceptualisation processes might be distinct. (shrink)

There is widespread interest in the possibility that music training enhances nonmusical abilities. This possibility has been examined primarily for speech perception and domain-general abilities su...

In the human-computer interaction, electroencephalogram access for automatic emotion recognition is an effective way for robot brains to perceive human behavior. In order to improve the accuracy of the emotion recognition, a method of EEG access for emotion recognition based on a deep hybrid network was proposed in this paper. Firstly, the collected EEG was decomposed into four frequency band signals, and the multiscale sample entropy features of each frequency band were extracted. Secondly, the (...) constructed 3D MSE feature matrices were fed into a deep hybrid network for autonomous learning. The deep hybrid network was composed of a continuous convolutional neural network and hidden Markov models. Lastly, HMMs trained with multiple observation sequences were used to replace the artificial neural network classifier in the CNN, and the emotion recognition task was completed by HMM classifiers. The proposed method was applied to the DEAP dataset for emotion recognition experiments, and the average accuracy could achieve 79.77% on arousal, 83.09% on valence, and 81.83% on dominance. Compared with the latest related methods, the accuracy was improved by 0.99% on valence and 14.58% on dominance, which verified the effectiveness of the proposed method. (shrink)

Tactile emotion recognition provides a lot of valuable information in human-computer interaction, and it has strong application prospects in many aspects such as smart home and medical treatment. So this situation raises a question: How to quickly and efficiently let the robot perform the correct emotion recognition? In this work, we develop a lifelong learning algorithm which is based on the efficient dictionary learning technology, to tackle the tactile emotion recognition across different tasks. To (...) verify the efficiency of the proposed method, we applied it to two data sets for experimentation: Corpus of Social Touch (CoST) and our dataset(We built it with a 12X12 array sensor). The results show that the proposed lifelong learning algorithm achieves satisfactory results. (shrink)

Emotion recognition plays an important part in human-computer interaction. Currently, the main challenge in electroencephalogram -based emotion recognition is the non-stationarity of EEG signals, which causes performance of the trained model decreasing over time. In this paper, we propose a two-level domain adaptation neural network to construct a transfer model for EEG-based emotion recognition. Specifically, deep features from the topological graph, which preserve topological information from EEG signals, are extracted using a deep neural network. (...) These features are then passed through TDANN for two-level domain confusion. The first level uses the maximum mean discrepancy to reduce the distribution discrepancy of deep features between source domain and target domain, and the second uses the domain adversarial neural network to force the deep features closer to their corresponding class centers. We evaluated the domain-transfer performance of the model on both our self-built data set and the public data set SEED. In the cross-day transfer experiment, the ability to accurately discriminate joy from other emotions was high: sadness, anger, and fear on the self-built data set. The accuracy reached 74.93% on the SEED data set. In the cross-subject transfer experiment, the ability to accurately discriminate joy from other emotions was equally high: sadness, anger, and fear on the self-built data set. The average accuracy reached 87.9% on the SEED data set, which was higher than WGAN-DA. The experimental results demonstrate that the proposed TDANN can effectively handle the domain transfer problem in EEG-based emotion recognition. (shrink)

According to simulation or shared-substrates models of emotion recognition, our ability to recognize the emotions expressed by other individuals relies, at least in part, on processes that internally simulate the same emotional state in ourselves. The term “emotional expressions” is nearly synonymous, in many people's minds, with facial expressions of emotion. However, vocal prosody and whole-body cues also convey emotional information. What is the relationship between these various channels of emotional communication? We first briefly review simulation models (...) of emotion recognition, and then discuss neuroscientific evidence related to these models, including studies using facial expressions, whole-body cues, and vocal prosody. We conclude by discussing these data in the context of simulation and shared-substrates models of emotion recognition. (shrink)

Recent studies of emotion mindreading reveal that for three emotions, fear, disgust, and anger, deficits in face-based recognition are paired with deficits in the production of the same emotion. What type of mindreading process would explain this pattern of paired deficits? The simulation approach and the theorizing approach are examined to determine their compatibility with the existing evidence. We conclude that the simulation approach offers the best explanation of the data. What computational steps might be used, however, (...) in simulation-style emotion detection? Four alternative models are explored: a generate-and-test model, a reverse simulation model, a variant of the reverse simulation model that employs an “as if” loop, and an unmediated resonance model. (shrink)

Emotion recognition plays an important role in intelligent human–computer interaction, but the related research still faces the problems of low accuracy and subject dependence. In this paper, an open-source software toolbox called MindLink-Eumpy is developed to recognize emotions by integrating electroencephalogram and facial expression information. MindLink-Eumpy first applies a series of tools to automatically obtain physiological data from subjects and then analyzes the obtained facial expression data and EEG data, respectively, and finally fuses the two different signals at (...) a decision level. In the detection of facial expressions, the algorithm used by MindLink-Eumpy is a multitask convolutional neural network based on transfer learning technique. In the detection of EEG, MindLink-Eumpy provides two algorithms, including a subject-dependent model based on support vector machine and a subject-independent model based on long short-term memory network. In the decision-level fusion, weight enumerator and AdaBoost technique are applied to combine the predictions of SVM and CNN. We conducted two offline experiments on the Database for Emotion Analysis Using Physiological Signals dataset and the Multimodal Database for Affect Recognition and Implicit Tagging dataset, respectively, and conducted an online experiment on 15 healthy subjects. The results show that multimodal methods outperform single-modal methods in both offline and online experiments. In the subject-dependent condition, the multimodal method achieved an accuracy of 71.00% in the valence dimension and an accuracy of 72.14% in the arousal dimension. In the subject-independent condition, the LSTM-based method achieved an accuracy of 78.56% in the valence dimension and an accuracy of 77.22% in the arousal dimension. The feasibility and efficiency of MindLink-Eumpy for emotion recognition is thus demonstrated. (shrink)

Emotions are multimodal processes that play a crucial role in our everyday lives. Recognizing emotions is becoming more critical in a wide range of application domains such as healthcare, education, human-computer interaction, Virtual Reality, intelligent agents, entertainment, and more. Facial macro-expressions or intense facial expressions are the most common modalities in recognizing emotional states. However, since facial expressions can be voluntarily controlled, they may not accurately represent emotional states. Earlier studies have shown that facial micro-expressions are more reliable than facial (...) macro-expressions for revealing emotions. They are subtle, involuntary movements responding to external stimuli that cannot be controlled. This paper proposes using facial micro-expressions combined with brain and physiological signals to more reliably detect underlying emotions. We describe our models for measuring arousal and valence levels from a combination of facial micro-expressions, Electroencephalography (EEG) signals, galvanic skin responses (GSR), and Photoplethysmography (PPG) signals. We then evaluate our model using the DEAP dataset and our own dataset based on a subject-independent approach. Lastly, we discuss our results, the limitations of our work, and how these limitations could be overcome. We also discuss future directions for using facial micro-expressions and physiological signals in emotion recognition. (shrink)