Confusion prediction from eye-tracking data: Experiments with machine learning

Joni Salminen; Haewoon Kwak; Soon Gyo Jung; Mridul Nagpal; Jisun An; Bernard J. Jansen

doi:10.1145/3361570.3361577

Confusion prediction from eye-tracking data: Experiments with machine learning

Joni Salminen, Haewoon Kwak, Soon Gyo Jung, Mridul Nagpal, Jisun An, Bernard J. Jansen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

Original language	English (US)
Title of host publication	Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450362924
DOIs	https://doi.org/10.1145/3361570.3361577
Publication status	Published - Mar 24 2019
Event	9th International Conference on Information Systems and Technologies, ICIST 2019 - Cairo, Egypt Duration: Mar 24 2019 → Mar 26 2019

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	9th International Conference on Information Systems and Technologies, ICIST 2019
Country	Egypt
City	Cairo
Period	3/24/19 → 3/26/19

Fingerprint

All Science Journal Classification (ASJC) codes

Human-Computer Interaction
Computer Networks and Communications
Computer Vision and Pattern Recognition
Software

Cite this

Salminen, J., Kwak, H., Jung, S. G., Nagpal, M., An, J., & Jansen, B. J. (2019). Confusion prediction from eye-tracking data: Experiments with machine learning. In Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019 [a5] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3361570.3361577

Salminen, Joni ; Kwak, Haewoon ; Jung, Soon Gyo ; Nagpal, Mridul ; An, Jisun ; Jansen, Bernard J. / Confusion prediction from eye-tracking data : Experiments with machine learning. Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery, 2019. (ACM International Conference Proceeding Series).

@inproceedings{0f7a387e9caf422aabc6d06ec6691b24,

title = "Confusion prediction from eye-tracking data: Experiments with machine learning",

abstract = "Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70{\%} when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90{\%}. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.",

author = "Joni Salminen and Haewoon Kwak and Jung, {Soon Gyo} and Mridul Nagpal and Jisun An and Jansen, {Bernard J.}",

year = "2019",

month = "3",

day = "24",

doi = "10.1145/3361570.3361577",

language = "English (US)",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

booktitle = "Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019",

note = "9th International Conference on Information Systems and Technologies, ICIST 2019 ; Conference date: 24-03-2019 Through 26-03-2019",

}

Salminen, J, Kwak, H, Jung, SG, Nagpal, M, An, J & Jansen, BJ 2019, Confusion prediction from eye-tracking data: Experiments with machine learning. in Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019., a5, ACM International Conference Proceeding Series, Association for Computing Machinery, 9th International Conference on Information Systems and Technologies, ICIST 2019, Cairo, Egypt, 3/24/19. https://doi.org/10.1145/3361570.3361577

Confusion prediction from eye-tracking data : Experiments with machine learning. / Salminen, Joni; Kwak, Haewoon; Jung, Soon Gyo; Nagpal, Mridul; An, Jisun; Jansen, Bernard J.

Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery, 2019. a5 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Confusion prediction from eye-tracking data

T2 - 9th International Conference on Information Systems and Technologies, ICIST 2019

AU - Salminen, Joni

AU - Kwak, Haewoon

AU - Jung, Soon Gyo

AU - Nagpal, Mridul

AU - An, Jisun

AU - Jansen, Bernard J.

PY - 2019/3/24

Y1 - 2019/3/24

N2 - Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

AB - Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

UR - http://www.scopus.com/inward/record.url?scp=85076139447&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076139447&partnerID=8YFLogxK

U2 - 10.1145/3361570.3361577

DO - 10.1145/3361570.3361577

M3 - Conference contribution

AN - SCOPUS:85076139447

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019

PB - Association for Computing Machinery

Y2 - 24 March 2019 through 26 March 2019

ER -

Salminen J, Kwak H, Jung SG, Nagpal M, An J, Jansen BJ. Confusion prediction from eye-tracking data: Experiments with machine learning. In Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery. 2019. a5. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3361570.3361577

Access to Document

10.1145/3361570.3361577