From mining affective states to mining facial keypoint data: The quest towards personalized feedback

Christian E. Lopez, Conrad S. Tucker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Personalized and timely feedback has the potential to improve an individual's performance on a wide variety of engineering tasks. The ability to capture an individual's affective state(s) and performance on a task is a key component needed to advance personalization of feedback. While automated methods exist for quantifying task performance, the ability to quantify an individual's affective state(s) remains an open research area. Existing methods for quantifying an individual's affective state(s) are challenging to implement where real-Time assessment is needed (e.g., engineering workshop environments). This has sparked a growing interest for automated systems capable of inferring individuals' affective state(s), based on their projected facial or body cues. However, existing methods attempt to employ a general model to label an individual's affective state(s) into discrete categories, such as fear, joy, surprise, etc. Nonetheless, emotional expressions are far more complex, as individual differences in facial expressions, may deteriorate the performance of these systems in providing personalized feedback. To overcome these limitations, this work proposes a machine learning method for predicting an individual's performance on a task by utilizing his/her unique facial keypoint data, hereby bypassing the need to infer his/her discrete affective states. A case study involving 31 participants is presented. The support vector machine model employed to predict an individual's performance yielded an accuracy of 77.15% for an individual-Task specific model. In contrast, a general model yielded an accuracy of only 52.69%, hereby supporting the authors' argument that individual-Task specific models are more suitable for advancing personalized feedback.

Original languageEnglish (US)
Title of host publication37th Computers and Information in Engineering Conference
PublisherAmerican Society of Mechanical Engineers (ASME)
ISBN (Electronic)9780791858110
DOIs
StatePublished - Jan 1 2017
EventASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2017 - Cleveland, United States
Duration: Aug 6 2017Aug 9 2017

Publication series

NameProceedings of the ASME Design Engineering Technical Conference
Volume1

Other

OtherASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2017
CountryUnited States
CityCleveland
Period8/6/178/9/17

All Science Journal Classification (ASJC) codes

  • Mechanical Engineering
  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Modeling and Simulation

Fingerprint Dive into the research topics of 'From mining affective states to mining facial keypoint data: The quest towards personalized feedback'. Together they form a unique fingerprint.

  • Cite this

    Lopez, C. E., & Tucker, C. S. (2017). From mining affective states to mining facial keypoint data: The quest towards personalized feedback. In 37th Computers and Information in Engineering Conference [67340] (Proceedings of the ASME Design Engineering Technical Conference; Vol. 1). American Society of Mechanical Engineers (ASME). https://doi.org/10.1115/DETC2017-67340