Search results

The purpose of this paper is to explore uncertainty inherent in emotion recognition technologies and the consequences resulting from that phenomenon.

The paper is a general overview of the concept; however, it is based on a meta-analysis of multiple experimental and observational studies performed over the past couple of years.

The main finding of the paper might be summarized as follows: there is uncertainty inherent in emotion recognition technologies, and the phenomenon is not expressed enough, not addressed enough and unknown by the users of the technology.

Practical implications of the study are formulated as postulates for the developers, users and researchers dealing with the technologies of automatic emotion recognition.

As technologies that recognize emotions are becoming more and more common, and perhaps more decisions influencing people lives are to come in the next decades, the trustworthiness of the technology is important from a scientific, practical and ethical point of view.

Studying uncertainty of emotion recognition technologies is a novel approach and is not explored from such a broad perspective before.

Uncertainty is an under-respected issue when it comes to automatic assessment of human emotion by machines. The purpose of this paper is to highlight the existent approaches towards such measurement of uncertainty, and identify further research need.

The discussion is based on a literature review.

Technical solutions towards measurement of uncertainty in automatic emotion recognition (AER) exist but need to be extended to respect a range of so far underrepresented sources of uncertainty. These then need to be integrated into systems available to general users.

Not all sources of uncertainty in automatic emotion recognition (AER) including emotion representation and annotation can be touched upon in this communication.

AER systems shall be enhanced by more meaningful and complete information provision on the uncertainty underlying their estimates. Limitations of their applicability should be communicated to users.

Users of automatic emotion recognition technology will become aware of their limitations, potentially leading to a fairer usage in crucial application context.

There is no previous discussion including the technical view point on extended uncertainty measurement in automatic emotion recognition.

Although numerous signal modalities are available for emotion recognition, audio and visual modalities are the most common and predominant forms for human beings to express their emotional states in daily communication. Therefore, how to achieve automatic and accurate audiovisual emotion recognition is significantly important for developing engaging and empathetic human–computer interaction environment. However, two major challenges exist in the field of audiovisual emotion recognition: (1) how to effectively capture representations of each single modality and eliminate redundant features and (2) how to efficiently integrate information from these two modalities to generate discriminative representations.

A novel key-frame extraction-based attention fusion network (KE-AFN) is proposed for audiovisual emotion recognition. KE-AFN attempts to integrate key-frame extraction with multimodal interaction and fusion to enhance audiovisual representations and reduce redundant computation, filling the research gaps of existing approaches. Specifically, the local maximum–based content analysis is designed to extract key-frames from videos for the purpose of eliminating data redundancy. Two modules, including “Multi-head Attention-based Intra-modality Interaction Module” and “Multi-head Attention-based Cross-modality Interaction Module”, are proposed to mine and capture intra- and cross-modality interactions for further reducing data redundancy and producing more powerful multimodal representations.

Extensive experiments on two benchmark datasets (i.e. RAVDESS and CMU-MOSEI) demonstrate the effectiveness and rationality of KE-AFN. Specifically, (1) KE-AFN is superior to state-of-the-art baselines for audiovisual emotion recognition. (2) Exploring the supplementary and complementary information of different modalities can provide more emotional clues for better emotion recognition. (3) The proposed key-frame extraction strategy can enhance the performance by more than 2.79 per cent on accuracy. (4) Both exploring intra- and cross-modality interactions and employing attention-based audiovisual fusion can lead to better prediction performance.

The proposed KE-AFN can support the development of engaging and empathetic human–computer interaction environment.

A vast body of literature has documented the negative consequences of stress on employee performance and well-being. These deleterious effects are particularly pronounced for service agents who need to constantly endure and manage customer emotions. The purpose of this paper is to introduce and describe a deep learning model to predict in real-time service agent stress from emotion patterns in voice-to-voice service interactions.

A deep learning model was developed to identify emotion patterns in call center interactions based on 363 recorded service interactions, subdivided in 27,889 manually expert-labeled three-second audio snippets. In a second step, the deep learning model was deployed in a call center for a period of one month to be further trained by the data collected from 40 service agents in another 4,672 service interactions.

The deep learning emotion classifier reached a balanced accuracy of 68% in predicting discrete emotions in service interactions. Integrating this model in a binary classification model, it was able to predict service agent stress with a balanced accuracy of 80%.

Service managers can benefit from employing the deep learning model to continuously and unobtrusively monitor the stress level of their service agents with numerous practical applications, including real-time early warning systems for service agents, customized training and automatically linking stress to customer-related outcomes.

The present study is the first to document an artificial intelligence (AI)-based model that is able to identify emotions in natural (i.e. nonstaged) interactions. It is further a pioneer in developing a smart emotion-based stress measure for service agents. Finally, the study contributes to the literature on the role of emotions in service interactions and employee stress.

The purpose of this study is to conduct a thematic content analysis of facial emotion recognition (FER) research within the context of the hospitality and tourism industry. Through this analysis, the study aims to identify key themes, trends and implications of the utilization of FER technology in enhancing customer emotions and experiences within hospitality and tourism settings.

This is qualitative research that utilizes thematic content analysis. The research data were obtained from the Scopus database. A total of 45 articles (titles, abstracts and keywords) were coded into MAXQDA and VOSWiever programs for data analyses and mapping.

Based on the analyses, the predominant term used in titles was emotion, indicating its centrality in the research domain. Moreover, the most prevalent concepts in this field were emotion and experience. Notably, facial emotion recognition emerged as the most frequently utilized term within this context. Within the hospitality and tourism industry, FER was primarily employed within the travel sub-branch. Finally, the research culminated in the visualization of the theoretical framework and conceptual background, offering a comprehensive overview of the field.

There is a growing demand for using FER technology specifically within the hospitality and tourism industry context; therefore, growing scientific research has been conducted on this topic recently. By conducting a thematic content analysis, this study uncovered novel insights into the utilization of this technology to enhance customer emotions and experiences, thereby contributing to a deeper understanding of its potential implications and applications within the hospitality and tourism industry.

The purpose of this paper is to bring together the textual and multimedia opinions, since the use of social data has become the new trend that enables to gather the product reputation traded in social media. Integrating a product reputation process into the companies' strategy will bring several benefits such as helping in decision-making regarding the current and the new generation of the product by understanding the customers’ needs. However, image-centric sentiment analysis has received much less attention than text-based sentiment detection.

In this work, the authors propose a multimedia content-based product reputation framework that helps in detecting opinions from social media. Thus, in this case, the analysis of a certain publication is made by combining their textual and multimedia parts.

To test the effectiveness of the proposed framework, a case study based on YouTube videos has been established, as it brings together the image, the audio and the video processing at the same time.

The key novelty is the implication of multimedia content in addition of the textual one with the goal of gathering opinions about a certain product. The multimedia analysis brings together facial sentiment detection, printed text analysis, opinion detection from speeches and textual opinion analysis.

This study aims to propose the effect of five sub-dimensions of the expected benefits, which include compatibility, social influence, convenience, function and emotion on attitude and behavioral intentions.

A research model including eight hypotheses was tested using 413 samples collected in South Korea.

The data analysis results indicated that the five sub-dimensions of expected benefits aid to enhance attitude, which plays an important role in the formation of behavioral intentions.

This study was designed to empirically identify the important role of expected benefits in the context of drone food delivery services for the first time.

无人机送餐服务的预期效益：对态度和行为意向影响的研究

本研究提出无人机送餐的预期效益, 及其五大维度（兼容性, 社交影响力, 方便, 实用, 以及情感）对消费者态度和行为意向的影响。

本研究样本包括413韩国消费者来检测提出的理论模型以及八项研究假设。

数据分析显示预期效益包括的五大维度可以提高消费者态度。消费者态度在促进行为意向产生了关键性影响。

本论文是有关预期效益在无人机送餐的关键性作用的首次实证研究

The purpose of this paper is to answer the question whether it is possible to recognise the gender of a web browser user on the basis of keystroke dynamics and mouse movements.

An experiment was organised in order to track mouse and keyboard usage using a special web browser plug-in. After collecting the data, a number of parameters describing the users’ keystrokes, mouse movements and clicks were calculated for each data sample. Then several machine learning methods were used to verify the stated research question.

The experiment showed that it is possible to recognise males and females on the basis of behavioural characteristics with an accuracy exceeding 70 per cent. The best results were obtained while using Bayesian networks.

The first limitation of the study was the restricted contextual information, i.e. neither the type of web page browsed nor the user activity was taken into account. Another is the narrow scope of the respondent group. Future work should focus on gathering data from more users covering a wider age range and should consider the context.

Automatic gender recognition could be used in profiling a user to create personalised websites or as an additional feature in automatic identification for security reasons. It might be also considered as a confirmation of declared gender in web-based surveys.

As not all users perceive personalised ads and websites as beneficial, this application requires the analysis of a user perspective to provide value to the consumer without privacy violation.

Behavioural characteristics, such as mouse movements and keystroke dynamics, have already been used for user authentication and emotion recognition, but applying these data to gender recognition is an original idea.

Nowadays, the speech emotion recognition (SER) model has enhanced as the main research topic in various fields including human–computer interaction as well as speech processing. Generally, it focuses on utilizing the models of machine learning for predicting the exact emotional status from speech. The advanced SER applications go successful in affective computing and human–computer interaction, which is making as the main component of computer system's next generation. This is because the natural human machine interface could grant the automatic service provisions, which need a better appreciation of user's emotional states.

This paper implements a new SER model that incorporates both gender and emotion recognition. Certain features are extracted and subjected for classification of emotions. For this, this paper uses deep belief network DBN model.

Through the performance analysis, it is observed that the developed method attains high accuracy rate (for best case) when compared to other methods, and it is 1.02% superior to whale optimization algorithm (WOA), 0.32% better from firefly (FF), 23.45% superior to particle swarm optimization (PSO) and 23.41% superior to genetic algorithm (GA). In case of worst scenario, the mean update of particle swarm and whale optimization (MUPW) in terms of accuracy is 15.63, 15.98, 16.06% and 16.03% superior to WOA, FF, PSO and GA, respectively. Under the mean case, the performance of MUPW is high, and it is 16.67, 10.38, 22.30 and 22.47% better from existing methods like WOA, FF, PSO, as well as GA, respectively.

This paper presents a new model for SER that aids both gender and emotion recognition. For the classification purpose, DBN is used and the weight of DBN is used and this is the first work uses MUPW algorithm for finding the optimal weight of DBN model.

Emotional appraisal is an act of sense making: What does a particular event mean for me? It is not the event itself – but rather an individual's subjective evaluation of the event – that elicits and shapes emotions (Scherer, 1997b). Thus, appraisal is the crucial first step in the emotion process, and describes how we attend, interpret and ascribe meaning to a given event or stimulus. First, emotional appraisal requires attention; given cognitive limits, we must prioritize which events are even worthy of our notice. Second, we must code the event, interpreting its meaning, and in particular its implications for the self (Mesquita & Frijda, 1992). If another person in a team environment is being rude, how one interprets the personal significance of this behavior may change significantly the emotional response – for example, whether the rude individual is a teammate, a customer, a supplier, or a competitor, and whether the rude behavior is directed at an innocent bystander or an instigator. Likewise, a bear approaching a campsite may elicit fear, but the same bear in a zoo could result in delight. Often the cognitive evaluation of stimuli associated with emotional appraisal occurs so quickly and automatically, before our conscious awareness, that we may be unaware of this individual component of the unfolding process. However, even in such cases, we can see the role of appraisal processes by examining, for example, how emotional reactions change over time and vary from person to person. An event that may have caused great embarrassment during youth might in adulthood leave one unfazed, and an event that makes one person angry might make another person sad. Indeed, it can be the lack of conscious awareness of the appraisal process – and the sense that appraisal is clear and lacking a subjective interpretive lens – that prevents individuals from questioning and evaluating it. This results in a particular challenge to reconciling colleagues’ often vastly differing emotional appraisals.