TapLogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors

Today's smartphones are shipped with various embedded motion sensors, such as the accelerometer, gyroscope, and orientation sensors. These motion sensors are useful in supporting the mobile UI innovation and motion-based commands. However, they also bring potential risks of leaking user's private information as they allow third party applications to monitor the motion changes of smartphones. In this paper, we study the feasibility of inferring a user's tap inputs to a smartphone with its integrated motion sensors. Specifically, we utilize an installed trojan application to stealthily monitor the movement and gesture changes of a smartphone using its on-board motion sensors. When the user is interacting with the trojan application, it learns the motion change patterns of tap events. Later, when the user is performing sensitive inputs, such as entering passwords on the touchscreen, the trojan application applies the learnt pattern to infer the occurrence of tap events on the touchscreen as well as the tapped positions on the touchscreen. For demonstration, we present the design and implementation of TapLogger, a trojan application for the Android platform, which stealthily logs the password of screen lock and the numbers entered during a phone call (e.g., credit card and PIN numbers). Statistical results are presented to show the feasibility of such inferences and attacks.