Project Objectives:

Tracking human vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., Polysomnography (PSG)) are limited to clinic usage. Recent radio frequency (RF) based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this project, we explore the possibility of tracking the vital signs (i.e., breathing rate and heart rate) during sleep by two alternative low cost ways: 1) using off-the-shelf WiFi without any wearable or dedicated devices; and 2) exploiting the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring.

Technology Rationale:

WiFi based Approach: While proliferating WiFi networks are usually used for wireless Internet access and connecting local area networks, such as an in-home WiFi network involving both mobile and stationary devices (e.g., laptop, smartphone, tablet, desktop, smartTV), they have great potential to sense the environment changes and capture the minute movements caused by human body. Indeed, WiFi signals are affected by human body movements at various scales during sleep, such as large scale movements involving going to bed and turn-over, minute movements including inhaling/exhaling for breathing and diastole/systole for heart beats. By extracting and analyzing the unique characteristics of WiFi signals, we could capture and derive the semantic meanings of such movements including both breathing rate and heart beats during sleep. We are thus motivated to re-use existing WiFi network to monitor the fine-grained vital signs during sleep as it doesn't require any dedicated/wearable sensors or additional infrastructure setup. Figure 1 shows the CSI amplitude of four subcarriers (i.e., subcarrier 1, 7, 19 and 28) extracted from a laptop in a 802.11n network over time when a person is asleep. His bed is in between an AP and the laptop with 3 meters apart. The person does not carry any sensor in his body. We observe that the CSI amplitude of these four subcarriers exhibits an obvious periodic up-and-down trend. Such a pattern could be caused by the person's breathing during sleep. This observation strongly suggest that we may achieve device-free fine-grained vital signs monitoring by leveraging the CSI from off-the-shelf WiFi devices.

Smartphone-based Approach: This work based on acoustic signals captured from smartphone earphone aims to achieve the fine-grained non-invasive sleep monitoring at a minimal cost by exploiting the off-the-shelf smartphone and its earphone. The proposed system captures the breathing sound generated by the air flow for breathing rate detection. The airway flow is correlated with the amplitude of the respiratory sound during normal breathing. It is thus possible to monitor the breathing rate based on the extracting the breathing sound from the smartphone earphone as shown in Figure 2. Moreover, the benefit of using earphone is four-fold. First, the microphone on earphone has a higher recording quality than that of the smartphone built-in microphone, resulting in more reliable recorded breathing sound. Second, many users are resistant to place the smartphone close to them during sleep but tend to leave the earphone plugged into their ears or put it aside on their pillows during sleep. Third, the earbuds on earphone could be used as microphones, which helps to enhance the recording ability of the breathing sound. Fourth, using earphone can also capture other sleep related events easily such as snoring, coughing, turn-over and get up.

Technical Approach:

WiFi based Approach: Our system uses only a single pair of WiFi device and wireless AP for detecting the breathing rate and heart rate during sleep. The breathing rate detection algorithm first obtains time series of CSI from off-the-shelf WiFi device (e.g., desktop, laptop, tablet, and smartphone) and then analyzes the information in time domain and frequency domain. It achieves high accuracy for both single and two-person in bed scenarios. To detect heart rate, our algorithm first applies a bandpass filter to eliminate irrelevant frequency components, and then estimates the heart rate in the frequency domain by locating the frequency peak in the normal heart rate range. Extensive experiments are conducted in lab environment and two apartments with different sizes. The results show that our system provides accurate breathing rate and heart rate estimation not only under typical settings but also covering challenging scenarios including long distance between the WiFi device and AP, none-line-of-sight (NLOS) situation and different sleep postures. This demonstrates that our approach can provide device-free, continuous fine-grained vital signs monitoring without any additional cost. It has the capability to support large-scale deployment and long-term vital signs monitoring in non-clinical settings.

Smart phone based Approach: The sleep monitoring using smartphone takes as input the recorded acoustic sound from the earphone. The sound can be further enhanced by using the input from the earbuds after connecting the earbuds with the output of the microphone by using off-the-shelf connecters. This system then performs noise estimation and subtraction to reduce the impact of background noise. The high correlation between a user's breathing cycles is exploited to make the breathing rate detection method adaptive to different users. Finally, the acoustic features extracted from acoustic sound are used for sleep event (e.g., snoring, coughing, turn-over and get up) detection. By combining breathing rate and sleep events, the proposed system can provide continuous and noninvasive fine-grained sleep monitoring for healthcare related applications (such as sleep apnea monitoring).

Results To Date and Future Work Plan:

This project has led to papers in ACM MobiHoc 2015 and IEEE INFOCOM 2015. The project is reported by MIT Technology Review, Fierce Mobile Healthcare, Digital Journal and Yahoo News, etc. Figure 3 shows the breathing rate estimation performance using off-the-shelf WiFi with different distances between WiFi devices and AP. Figure 4 presents the sample results of using smartphone earphone to detect breathing rate during sleep.