Microexpression Identification and Categorization Using a Facial Dynamics Map

Feng Xu, Junping Zhang, James Z. Wang

Research output: Contribution to journalArticlepeer-review

94 Scopus citations

Abstract

Unlike conventional facial expressions, microexpressions are instantaneous and involuntary reflections of human emotion. Because microexpressions are fleeting, lasting only a few frames within a video sequence, they are difficult to perceive and interpret correctly, and they are highly challenging to identify and categorize automatically. Existing recognition methods are often ineffective at handling subtle face displacements, which can be prevalent in typical microexpression applications due to the constant movements of the individuals being observed. To address this problem, a novel method called the Facial Dynamics Map is proposed to characterize the movements of a microexpression in different granularity. Specifically, an algorithm based on optical flow estimation is used to perform pixel-level alignment for microexpression sequences. Each expression sequence is then divided into spatiotemporal cuboids in the chosen granularity. We also present an iterative optimal strategy to calculate the principal optical flow direction of each cuboid for better representation of the local facial dynamics. With these principal directions, the resulting Facial Dynamics Map can characterize a microexpression sequence. Finally, a classifier is developed to identify the presence of microexpressions and to categorize different types. Experimental results on four benchmark datasets demonstrate higher recognition performance and improved interpretability.

Original languageEnglish (US)
Article number7383236
Pages (from-to)254-267
Number of pages14
JournalIEEE Transactions on Affective Computing
Volume8
Issue number2
DOIs
StatePublished - Apr 1 2017

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Microexpression Identification and Categorization Using a Facial Dynamics Map'. Together they form a unique fingerprint.

Cite this