Search results

This study aims to introduce an affective computing‐based method of identifying student understanding throughout a distance learning course.

The study proposed a learning emotion recognition model that included three phases: feature extraction and generation, feature subset selection and emotion recognition. Features are extracted from facial images and transform a given measument of facial expressions to a new set of features defining and computing by eigenvectors. Feature subset selection uses the immune memory clone algorithms to optimize the feature selection. Emotion recognition uses a classifier to build the connection between facial expression and learning emotion.

Experimental results using the basic expression of facial expression recognition research database, JAFFE, show that the proposed facial expression recognition method has high classification performance. The experiment results also show that the recognition of spontaneous facial expressions is effective in the synchronous distance learning courses.

The study shows that identifying student comprehension based on facial expression recognition in synchronous distance learning courses is feasible. This can help instrutors understand the student comprehension real time. So instructors can adapt their teaching materials and strategy to fit with the learning status of students.

Various studies have examined the role of executive functions in autism, but there is a lack of research in the current literature on cognitive flexibility in autism spectrum disorders (ASD). The purpose of this study is to investigate whether cognitive flexibility deficits could be related to facial emotion recognition deficits in ASD.

In total, 20 children with ASD and 20 typically developing children, matched for intelligence quotient and gender, were examined both in facial emotion recognition tasks and in cognitive flexibility tasks through the dimensional change card sorting task.

Despite cognitive flexibility not being a core deficit in ASD, impaired cognitive flexibility is evident in the present research. Results show that cognitive flexibility is related to facial emotion recognition and support the hypothesis of an executive specific deficit in children with autism.

One of the limit is the use of just one cognitive test to measure cognitive flexibility and facial recognition. This could be important to be taken into account in the new research. By increasing the number of common variables assessing cognitive flexibility, this will allow for a better comparison between studies to characterize impairment in cognitive flexibility in ASD.

Investigating impairment in cognitive flexibility may help to plan training intervention based on the induction of flexibility.

If the authors implement cognitive flexibility people with ASD can have also an effect on their social behavior and overcome the typical and repetitive behaviors that are the hallmark of ASD.

The originality is to relate cognitive flexibility deficits to facial emotion.

Data-driven marketing is replacing conventional marketing strategies. The modern marketing strategy is based on insights derived from customer behavior information gathered from their facial expressions and neuro-signals. This study explores the potential for face recognition and neuro-marketing in modern-day marketing.

The study conducts an in-depth examination of the extant literature on neuro-marketing and facial recognition marketing. The articles for review are downloaded from the Scopus database, and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is then used to screen and choose the relevant papers. The systematic literature review method is applied to conduct the study.

An extensive review of the literature reveals that the domains of neuro-marketing and face recognition marketing remain understudied. The authors’ review of selected papers delivers five neuro-marketing and facial recognition marketing themes that are essential to modern marketing concepts.

Neuro-marketing and facial recognition marketing are artificial intelligence (AI)-enabled marketing techniques that assist in gaining cognitive insights into human behavior. The findings would be of use to managers in designing marketing strategies to enhance their marketing approach and boost conversion rates.

The uniqueness of this study lies in that it provides an updated review on neuro-marketing and face recognition marketing.

The purpose of this paper is to explore uncertainty inherent in emotion recognition technologies and the consequences resulting from that phenomenon.

The paper is a general overview of the concept; however, it is based on a meta-analysis of multiple experimental and observational studies performed over the past couple of years.

The main finding of the paper might be summarized as follows: there is uncertainty inherent in emotion recognition technologies, and the phenomenon is not expressed enough, not addressed enough and unknown by the users of the technology.

Practical implications of the study are formulated as postulates for the developers, users and researchers dealing with the technologies of automatic emotion recognition.

As technologies that recognize emotions are becoming more and more common, and perhaps more decisions influencing people lives are to come in the next decades, the trustworthiness of the technology is important from a scientific, practical and ethical point of view.

Studying uncertainty of emotion recognition technologies is a novel approach and is not explored from such a broad perspective before.

The purpose of this paper is to examine the associations among psychopathic and borderline traits, intimate partner violence (IPV) and sensitivity to facial affect. The authors hypothesized that IPV men high in psychopathic traits would exhibit reduced sensitivity to expressions of distress specifically (fear + sadness), while IPV men high in borderline traits would show heightened sensitivity to facial affect more generally.

A community sample of 79 IPV men in heterosexual relationships were exposed to slides of facial affect displays while psychophysiological reactions were recorded. Sensitivity to facial affect was operationalized as accuracy in recognizing and skin conductance responses (SCR) while viewing discrete facial expressions.

Borderline personality disorder (BPD) features were positively related to accuracy in labeling fear and surprise while primary psychopathy (Factor 1) was negatively related to accuracy in labeling disgust. Borderline traits were positively associated with SCR while primary psychopathy was negatively associated with SCR while viewing slides of facial affect. Secondary psychopathy (Factor 2) follows the same physiological patterns of BPD traits but the correlates are weaker. Results suggest that IPV men high in traits of primary psychopathy show hypoarousal whereas those high borderline features show hyperarousal to facial emotions.

Limitations include a small sample of heterosexual violent community couples. Women’s IPV was not analyzed. Findings suggest that BPD and primary psychopathy traits are diametrically opposite in SCR, making them powerful comparison groups for psychophysiological studies. Findings challenge Blair’s (1995) model of a specific deficit in processing distress cues for individuals high in psychopathic traits. Rather results suggest that IPV men high in traits of primary psychopathy show more pervasive hypoarousal to facial emotion. The hyperarousal of men high in BPD traits across facial expressions supports Linehan’s (1993) emotional vulnerability model of borderline personality disorder.

Differences in psychophysiological responding to emotions may be clinically relevant in the motivations for violence perpetration. The hypoarousal associated with primary psychopathy may facilitate the perpetration of proactive violence. The hyperarousal associated with BPD and secondary psychopathy may be fundamental in the perpetration of reactive violence. Treatment matching by IPV perpetrators’ personality traits may improve the efficacy of battering intervention programs. Perpetrators high in borderline personality features may benefit from emotional regulation therapies, such as Dialectical Behavior Therapy. IPV men high in traits of primary psychopathy may benefit from affective empathy and validation training.

Currently, battering intervention programs show little efficacy in reducing intimate partner recidivism. Experimental psychopathology studies such as this one may inform advocates seeking to develop new, tailored treatment packages for partner violence offenders with different personality disorder traits.

Many treatment providers assume that men who batter women have deficits in empathy and emotional intelligence. However, this study suggests that rather than global deficits, deficits depend on personality traits. The current study is the first to assess psychophysiological reactivity in response to facial affect displays among IPV perpetrators. Examining SCR responding to photos of facial affect may be used in future studies of affect sensitivity.

This study aims to test service providers’ ability to recognise non-verbal emotions in complaining customers of same and different cultures.

In a laboratory study, using a between-subjects experimental design (n = 153), we tested the accuracy of service providers’ perceptions of the emotional expressions of anger, fear, shame and happiness of customers from varying cultural backgrounds. After viewing video vignettes of customers complaining (with the audio removed), participants (in the role of service providers) assessed the emotional state of the customers portrayed in the video.

Service providers in culturally mismatched dyads were prone to misreading anger, happiness and shame expressed by dissatisfied customers. Happiness was misread in the displayed emotions of both dyads. Anger was recognisable in the Anglo customers but not Confucian Asian, while Anglo service providers misread both shame and happiness in Confucian Asian customers.

The study was conducted in the laboratory and was based solely on participant’s perceptions of actors’ non-verbal facial expressions in a single encounter.

Given the level of ethnic differences in developed nations, a culturally sensitive workplace is needed to foster effective functioning of service employee teams. Ability to understand cultural display rules and to recognise and interpret emotions is an important skill for people working in direct contact with customers.

This research addresses the lack of empirical evidence for the recognition of customer emotions by service providers and the impact of cross-cultural differences.

The purpose of this paper is to investigate an idea of producing an assistive and augmentative communication (AAC) tool that uses natural human computer interfacing to accommodate the disabilities of children with cerebral palsy (CP) and assist them in their daily communication.

The authors developed a prototype that recognizes the real time detected emotions display on the face and send alerts to the caretakers through Short Messaging System (SMS) or loud speaker.

The evaluation result shows that the proposed prototype recognizes real time facial expressions from the children with CP with an average of 79.4 per cent, and a maximum of 88.3 per cent (standard deviation of 7.4 per cent) on ten children with CP. Evaluations were also conducted to investigate the effectiveness of the prototype to deliver critical expression messages to their caretakers. The result showed that 98.5 per cent of SMS was sent successfully to the caretakers (pre‐defined mobile phone number) with an average waiting time of 8.3 seconds.

The paper demonstrates the potential of the proposed prototype to assist children with CP to communicate with their caretakers in real time.

This paper aims to clarify how affect-based adaptation can improve implicit recognition of playing style of individuals during game sessions. This study presents the “Rush for Gold” game using dynamic difficulty adjustment of tasks based on both player performance and affectation inferred through electrodermal activity and facial expressions of the player. The game applies linear regression for calculating playing styles to be applied for achieving a style-based adaptation in other educational video games.

The experimental procedure included subject selection, demonstration, informed consent procedure, two game sessions in random order – one without and another with affective adaptation control – and post-game self-report. The experiment was conducted with participation of 30 master students and university lecturers in informatics.

This study presents experimental results concerning the impact of affective adaptation over playing style recognition, game session time, task’s effectiveness, efficiency and difficulty and, as well, player’s assessment of affectively adaptive gameplay obtained by an adaptation control panel embedded into the game and by post-game self-report.

The proposed adaptive game limits recognised styles to such based on the Kolb’s Learning Style Inventory model. Another limitation of the study is the relatively small number of participants constrained by the extended experimental procedure and the desktop game version.

The paper presents an original research on the effect of affect-based adaptation on a novel approach for implicit recognition of playing styles.

recent years, facial expression recognition has been widely used in human machine interaction, clinical medicine and safe driving. However, there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.

To solve such limitation, this paper proposes a novel model based on bidirectional gated recurrent unit networks (Bi-GRUs) with two-way propagations, and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network. Since the Inception-V3 network model for spatial feature extraction has too many parameters, it is prone to overfitting during training. This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters, so as to obtain an Inception-W network with better generalization.

Finally, the proposed model is pretrained to determine the best settings and selections. Then, the pretrained model is experimented on two facial expression data sets of CK+ and Oulu- CASIA, and the recognition performance and efficiency are compared with the existing methods. The highest recognition rate is 99.6%, which shows that the method has good recognition accuracy in a certain range.

By using the proposed model for the applications of facial expression, the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

Although numerous signal modalities are available for emotion recognition, audio and visual modalities are the most common and predominant forms for human beings to express their emotional states in daily communication. Therefore, how to achieve automatic and accurate audiovisual emotion recognition is significantly important for developing engaging and empathetic human–computer interaction environment. However, two major challenges exist in the field of audiovisual emotion recognition: (1) how to effectively capture representations of each single modality and eliminate redundant features and (2) how to efficiently integrate information from these two modalities to generate discriminative representations.

A novel key-frame extraction-based attention fusion network (KE-AFN) is proposed for audiovisual emotion recognition. KE-AFN attempts to integrate key-frame extraction with multimodal interaction and fusion to enhance audiovisual representations and reduce redundant computation, filling the research gaps of existing approaches. Specifically, the local maximum–based content analysis is designed to extract key-frames from videos for the purpose of eliminating data redundancy. Two modules, including “Multi-head Attention-based Intra-modality Interaction Module” and “Multi-head Attention-based Cross-modality Interaction Module”, are proposed to mine and capture intra- and cross-modality interactions for further reducing data redundancy and producing more powerful multimodal representations.

Extensive experiments on two benchmark datasets (i.e. RAVDESS and CMU-MOSEI) demonstrate the effectiveness and rationality of KE-AFN. Specifically, (1) KE-AFN is superior to state-of-the-art baselines for audiovisual emotion recognition. (2) Exploring the supplementary and complementary information of different modalities can provide more emotional clues for better emotion recognition. (3) The proposed key-frame extraction strategy can enhance the performance by more than 2.79 per cent on accuracy. (4) Both exploring intra- and cross-modality interactions and employing attention-based audiovisual fusion can lead to better prediction performance.

The proposed KE-AFN can support the development of engaging and empathetic human–computer interaction environment.