To read this content please select one of the options below:

Using AI to predict service agent stress from emotion patterns in service interactions

Stefano Bromuri (Computer Science, Open University of the Netherlands, Heerlen, The Netherlands)
Alexander P. Henkel (Organization, Open University of the Netherlands, Heerlen, The Netherlands)
Deniz Iren (Information Systems, Open University of the Netherlands, Heerlen, The Netherlands)
Visara Urovi (Institute of Data Science, Maastricht University, Maastricht, The Netherlands)

Journal of Service Management

ISSN: 1757-5818

Article publication date: 29 September 2020

Issue publication date: 10 June 2021




A vast body of literature has documented the negative consequences of stress on employee performance and well-being. These deleterious effects are particularly pronounced for service agents who need to constantly endure and manage customer emotions. The purpose of this paper is to introduce and describe a deep learning model to predict in real-time service agent stress from emotion patterns in voice-to-voice service interactions.


A deep learning model was developed to identify emotion patterns in call center interactions based on 363 recorded service interactions, subdivided in 27,889 manually expert-labeled three-second audio snippets. In a second step, the deep learning model was deployed in a call center for a period of one month to be further trained by the data collected from 40 service agents in another 4,672 service interactions.


The deep learning emotion classifier reached a balanced accuracy of 68% in predicting discrete emotions in service interactions. Integrating this model in a binary classification model, it was able to predict service agent stress with a balanced accuracy of 80%.

Practical implications

Service managers can benefit from employing the deep learning model to continuously and unobtrusively monitor the stress level of their service agents with numerous practical applications, including real-time early warning systems for service agents, customized training and automatically linking stress to customer-related outcomes.


The present study is the first to document an artificial intelligence (AI)-based model that is able to identify emotions in natural (i.e. nonstaged) interactions. It is further a pioneer in developing a smart emotion-based stress measure for service agents. Finally, the study contributes to the literature on the role of emotions in service interactions and employee stress.



The authors would like to thank the participating companies for providing access to their data as well as all service agents for their dedication in coding the customer service interactions.Funding: This research was supported by the Province of Limburg, The Netherlands, under grant number SAS-2019-00247.


Bromuri, S., Henkel, A.P., Iren, D. and Urovi, V. (2021), "Using AI to predict service agent stress from emotion patterns in service interactions", Journal of Service Management, Vol. 32 No. 4, pp. 581-611.



Emerald Publishing Limited

Copyright © 2020, Emerald Publishing Limited

Related articles