Automatic monitoring of facial emotions to measure deception

Description

Currently, there is a trend in deception research whereby researchers are using technology more often to avoid the reliability and potential bias from human coding. One of these technologies is Facial Expression Analysis (FEA). The software is able to pick-up muscle activations of the face. Certain combined movements of these facial muscles pertain to a displayed emotion. This means that we can automatically record and analyse the facial expressions of someone to estimate felt emotion, and test whether these change when receiving deceptive communications compared to honest communication.

A combination of data gathered from the software and previous research into dominant emotions in deception can give interesting insights. For example, the quick initial emotion of an interviewer is argued to be a more reliable predictor of veracity, than deliberate deception judgement. In other words, we might non-consciously identify emotion that we fail to acknowledge consciously when asked directly whether the person we communicate with is lying to us.

Research questions

This project will test the FEA system by determining whether it can reliably identify different emotional expressions in people receiving deceptive versus honest communication; and whether such differences are in the emotion displayed (e.g. positive vs negative emotions) or the match between sender and receiver (i.e. mimicry).

RESEARCH method

The study will be in the form of an experiment. The BMS Lab will assist with working with the software.

KEYWORDS

Deception; facial expressions; mimicry; observer; facial expression analysis

INFORMATION

Please contact Lynn Weiher (l.weiher@utwente.nl) when you are interested in this assignment. The assignment is open to one student.

LITERATURE 

Delmas, H., Elissalde, B., Rochat, N., Demarchi, S., Tijus, C., & Urdapilleta, I. (2019). Policemen’s and Civilians’ Beliefs About Facial Cues of Deception. Journal of Nonverbal Behavior , 43, Springer US. https://doi.org/10.1007/s10919-018-0285-4

Kulke, L., Feyerabend, D., & Schacht, A. (2020). A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion. Frontiers in Psychology, 11, 1–9. https://doi.org/10.3389/fpsyg.2020.00329

Porter, S., & Brinke, L. Ten. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science19(5), 508–514.