TY - GEN
T1 - Using screenshots to predict task switching on smartphones
AU - Yang, Xiao
AU - Robinson, Thomas
AU - Ram, Nilam
AU - Reeves, Byron
N1 - Publisher Copyright:
© 2019 Copyright held by the owner/author(s).
PY - 2019/5/2
Y1 - 2019/5/2
N2 - Mobile phone use is pervasive, yet little is known about task switching on digital platforms and applications. We propose an unobtrusive experience sampling method to observe how individuals use their smartphones by taking screenshots every 5 seconds when the device is on. The purpose of this paper is to incorporate the psychological process into feature extraction, and use these features to effectively predict the task switching behavior on smartphones. Features are extracted from the sequence of screenshots, gauging visual stimulation, cognitive load, velocity and accumulation, sentiment, and time-related factors. Labels of task switching behavior were manually tagged for 87,182 screenshots from 60 subjects. Using random forest, we demonstrate that we can correctly infer a user's task switching behavior from unstructured data in screenshots with up to 77% accuracy, demonstrating it is a viable option to use features of the screenshots to predict task switching behavior.
AB - Mobile phone use is pervasive, yet little is known about task switching on digital platforms and applications. We propose an unobtrusive experience sampling method to observe how individuals use their smartphones by taking screenshots every 5 seconds when the device is on. The purpose of this paper is to incorporate the psychological process into feature extraction, and use these features to effectively predict the task switching behavior on smartphones. Features are extracted from the sequence of screenshots, gauging visual stimulation, cognitive load, velocity and accumulation, sentiment, and time-related factors. Labels of task switching behavior were manually tagged for 87,182 screenshots from 60 subjects. Using random forest, we demonstrate that we can correctly infer a user's task switching behavior from unstructured data in screenshots with up to 77% accuracy, demonstrating it is a viable option to use features of the screenshots to predict task switching behavior.
UR - http://www.scopus.com/inward/record.url?scp=85067299569&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85067299569&partnerID=8YFLogxK
U2 - 10.1145/3290607.3313089
DO - 10.1145/3290607.3313089
M3 - Conference contribution
AN - SCOPUS:85067299569
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
T2 - 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019
Y2 - 4 May 2019 through 9 May 2019
ER -