Project Objectives:

This project explores whether a user's personal PIN sequence could be leaked through his wearable devices (e.g., smartwatch or fitness tracker), when accessing a key-based security system. Such systems are very common in daily lives. Examples include accessing ATM cash machines, electronic door locks, and keypad-controlled enterprise servers. A key-based security system requires people to enter personal key combinations on the keypad for identity verification. With people tending to wear wearable devices around-the-clock, the movements of their wrists during the key entry process to a security system (i.e., clicking keys and moving between clicks) are captured by the sensors on wearable devices. As such, wearables could cause a new way of sensitive information leakage when a user accesses the key-based security systems. The objective of this project is to explore the possibility of using embedded sensors in wearable devices, i.e., accelerometers, gyroscopes, and magnetometers, to derive the moving trajectories of the user's hand between consecutive key entries and further infer the user's PIN number.

Technology Rationale:

This project aims to develop a training-free, context-free technique to reveal a user's private PIN sequence (to a key-based security system) when a wrist-worn wearable device is employed. The wrist-worn wearable devices could be either smartwatches or fitness trackers. We consider an adversary aiming at recovering a victim's secret PIN entries leveraging embedded sensors (e.g., accelerometer, gyroscope and magnetometer) in wearable devices worn on the victim's wrist. The sensor data can be obtained by the adversary from two representative attacking scenarios, sniffing attacks [1] and internal attacks [2]. The obtained sensors readings from the wearable can be further used to capture dynamics of key entry activities. As shown in Figure2, the unique up-and-down acceleration pattern captured by the sensor when the user moves the finger from one key to another can be utilized to determine the sensor's moving direction and the accelerations during this movement can be double integrated to estimate the moving distance. The recovered fine-grained hand movement trajectories from the estimated directions and distances can then be utilized to derive the user's secret key entries.

Technical Approach:

Our approach examines the inherent physics phenomenon extracted from the user's key entry activities via wearable sensors and develops distance calculation and direction derivation schemes to produce mm-level accuracy when estimating the moving distance and angle between two consecutive key entries. To obtain the complete PIN sequence, our backward PIN-sequence inference algorithm exploits the physical constraints of distance between keys and temporal sequence of key entry activities to construct a tree of candidate key entries for determining the PIN sequence in a reversed manner, because in many practical cases, the "Enter" key is the last key after the user enters his/her PIN sequence. The mm-level precision of estimating the fine-grained moving distance and direction between two keys and the backward PIN-sequence inference algorithm enable our system to obtain the user's PIN sequence without training and contextual information. Such a technique can also be extended to support password recovery when people type on keyboards while wearing wearables.

Results To Date and Future Work Plan:

This project has led to the papers in AsiaCCS 2016 and Journal TMC 2017 and has received the Best Paper Award in AsiaCCS 2016. This work has been reported by over 60 media outlets, e.g., FORTUNE, PHYS, IEEE Spectrum and live interview with Top of Mind with Julie Rose on BYU Radio. We report the evaluation of our system with three different kinds of keypads and three different types of wearable devices and the sample results in Figure 2 show the top-k success rate of revealing the PINs.