Project Objectives:

The objective of this project is to search for an integrated solution that can perform systematic fitness monitoring and performance review to users. This project designs and implements a system that has the ability to automatically perform personalized workout interpretation and smart workout assessment. The proposed system aims to provide personalized fitness assistance to the users who perform exercise at gym, home and office environment.

Technology Rationale:

1) Mobile devices. A key observation is that most regular exercises involve repetitive arm movements. Such repetitive arm movements result in regularly changing values in sensor readings. In addition, the repetitive patterns from exercises tend to be last for a long time period simply because people normally adopt a set-and-rep scheme in exercise to maximize the effectiveness.

2) WiFi signal. Because of the prevalence of WiFi infrastructures in home/office environments, it is possible to capture people's activities without their participation. Given the inherent characteristics of a workout, we anticipate that we can observe repetitive CSI patterns because of the repetitive movements from different body parts during workouts.

Technical Approach:

1) Mobile devices. Given that wearable mobile devices are worn on the human body of either wrist (e.g., smartwatch) or upper arm (e.g., smartphone) during exercise, they become desirable interfaces to sense exercise movements to provide detailed workout statistics/analysis. We found that the repetitive pattern of body movements in exercises can be well captured by using the inertial sensors of the wearable mobile device. We then exploit Short Time Energy (STE) along with a SVM-based light-weight classifier to provide fine-grained workout interpretation including workout types and statistic analysis (i.e., the number of repetitions and sets)

2) WiFi signals. To further release the requirement of attaching devices, we find that the prevalence of WiFi infrastructures in home/office environment make WiFi signal a desirable candidate to facilitate workout interpretation and workout recommendation. In particular, our system leverage Channel State Information (CSI) and performs individual identification via deep learning techniques on top of workout interpretation. It further assesses the workout by analyzing both short and long-term workout quality via spectrogram analysis, and provides workout reviews for users to improve their daily exercises.

Project Status:

This project has led to one paper in INFOCOM'17. We report the performance and robustness of our system using both smartwatch and smartphone during people's workout. Figure 1 depicts the STE analysis on accelerometer readings from mobile, and Figure 2 shows the exercise recognition accuracy by using mobile devices.

We also have some initial results using WiFi signal. In Figure 3, we show the spectrogram analysis on WiFi when a person performs four sets, five repetitions per set, of one exercise. In addition, our system achieves high accuracy in both exercise recognition and people identification by using WiFi signal as shown in Figure 4 and Figure 5 respectively.

We will work on more complex environments with multiple people involved at the same time.

1) Mobile devices

2) WiFi signal