Google has conducted research on using smartphone technology to passively measure heart rate during regular smartphone usage. Resting heart rate (RHR) serves as a biomarker for cardiovascular health, with elevated RHR linked to adverse cardiovascular events and chronic illnesses. With around five billion smartphone owners globally, the potential for enhanced health tracking is significant.
Read More
In 2022, Google showcased how heart rate could be measured by placing a finger over the phone camera. Now, its new research introduces passive heart rate monitoring (PHRM) that tracks heart rate and RHR in the background while the smartphone is in use. This method employs the front-facing camera to capture video of the user’s face, utilizing deep learning to estimate heart rate with a mean absolute percentage error of less than 10%, meeting industry accuracy standards for individuals of all skin tones.
Eric S. Teasley, Product Manager, and Ming-Zer Poh, Staff Research Scientist at Google Research, stated that PHRM represents the first extensive demonstration of passive heart rate and daily RHR monitoring integrated into everyday smartphone activities. They highlighted that it not only adheres to heart rate accuracy standards for all skin tones but also sets a new benchmark for the remote photoplethysmography (rPPG) field, offering a wearable-like heart health insight through smartphone usage.
PHRM measures heart rate by detecting how light interacts with skin as blood pulses through it. An on-device software pipeline processes short facial video clips, using temporal shift convolutional neural networks to predict heart rate. Traditional studies have often overlooked individuals with darker skin tones due to the challenges that melanin presents in signal detection. Google's research used over 350,000 video clips from nearly 700 diverse participants, ensuring a balanced representation across light, medium, and dark skin tones, making it the most extensive and inclusive rPPG study conducted to date.
To train the system, Google recorded facial videos and electrocardiogram (ECG) data from 365 participants in controlled laboratory settings. The PHRM software demonstrated significantly superior performance compared to 15 leading rPPG models. Additionally, a separate free-living study involved 231 participants who utilized a custom data collection app while wearing ECG chest straps and Fitbit trackers, resulting in an average of 231 video clips recorded per day. Future research will aim to optimize camera exposure and improve accuracy during excessive head movements. Google plans to encourage further investigation by making its data and modeling resources available to qualified researchers.