To read the full version of this content please select one of the options below:

Modeling observer happiness from facial hyperspectral sensor

Min Hao (College of Electronic and Information Engineering, Southwest University, Chongqing, China)
Guangyuan Liu (College of Electronic and Information Engineering, Southwest University, Chongqing, China)
Desheng Xie (iFLYTEK Co., Ltd., China)
Ming Ye (College of Computer and Information Science, Southwest University, Chongqing, China)
Jing Cai (Center of Technical Support for Network Security, Chongqing, China)

Engineering Computations

ISSN: 0264-4401

Article publication date: 27 August 2019

Issue publication date: 16 January 2020



Happiness is an important mental emotion and yet becoming a major health concern nowadays. For this reason, better recognizing the objective understanding of how humans respond to event-related observations in their daily lives is especially important.


This paper uses non-intrusive technology (hyperspectral imaging [HSI]) for happiness recognition. Experimental setup is conducted for data collection in real-life environments where observers are showing spontaneous expressions of emotions (calm, happy, unhappy: angry) during the experimental process. Based on facial imaging captured from HSI, this work collects our emotional database defined as SWU Happiness DB and studies whether the physiological signal (i.e. tissue oxygen saturation [StO2], obtained by an optical absorption model) can be used to recognize observer happiness automatically. It proposes a novel method to capture local dynamic patterns (LDP) in facial regions, introducing local variations in facial StO2 to fully use physiological characteristics with regard to hyperspectral patterns. Further, it applies a linear discriminant analysis-based support vector machine to recognize happiness patterns.


The results show that the best classification accuracy is 97.89 per cent, objectively demonstrating a feasible application of LDP features on happiness recognition.


This paper proposes a novel feature (i.e. LDP) to represent the local variations in facial StO2 for modeling the active happiness. It provides a possible extension to the promising practical application.



This research is supported by the National Natural Science Foundation of China (Grant No. 61872301), Chongqing Key Laboratory of Non-linear Circuit and Intelligent Information Processing and Fundamental Research Funds for the Central Universities (XDJK2019C021). The authors are grateful to their colleagues from the Institute of Signal and Information Processing at Southwest University for their thoughtful suggestions. M. Hao would like to appreciate the helpful comments and reviews of Professor G. Liu in particular. M. Hao also acknowledges the help of D. Xie for his technical suggestions and M. Ye and J. Cai for their reviews and feedback in the early versions of this manuscript. Last, but not least, the authors would like to thank the anonymous reviewers for their constructive comments and suggestions.

Conflict of interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Hao, M., Liu, G., Xie, D., Ye, M. and Cai, J. (2020), "Modeling observer happiness from facial hyperspectral sensor", Engineering Computations, Vol. 37 No. 1, pp. 161-180.



Emerald Publishing Limited

Copyright © 2019, Emerald Publishing Limited