Consumer grade smartwatches may not be as accurate as promised when measuring tiredness or stress. That conclusion is drawn by researchers Björn Siepe and Eiko Fried based on a comparison between smartwatch measurements and reporting by users themselves.
It is impossible to imagine life without the smartwatch for a huge group of people. About 455 million consumers worldwide used a smartwatch in 2024. They are especially popular among young adults (18 - 34 years old); in this age group, about 40% use a smartwatch. Among other things, the smartwatch measures your heart rate and, based on biometric data, gives you information about how well you sleep, how tired or how stressed you are. But especially for the latter two topics, fatigue and stress, new research shows that it is questionable whether smartwatches are accurate.
Smartwatch vs self-reporting
The research team of Eiko Fried, led by Björn Siepe, investigated the use of wearable sensor data (from smartwatches) alongside smartphone-based self-reporting (EMA, ecological momentary assessment) as part of a bigger research programme on building WARN-D, an early warning system for depression. For 3 months, they collected data from nearly 800 smartwatch users and also queried them 4 times per day to fill out short questions on their smartphones about their current thoughts, feelings, and activities. In the evening, participants reflected on their day, for instance about their best or worst experiences.
The team then focused on the overlap between collected wearable (physiological) data and EMA data core constructs like stress, tiredness, and sleep. Their data shows the overlap is minimal, especially for measuring stress. In other words: when wearables reported that their owners were experiencing stress, the owners themselves rarely reported feeling stressed.
Based on a heart beat
One of the reasons for this lack of overlap, argues Fried, is that wrist-worn devices infer stress mostly based on heart rate and its variability. 'But changes of heart rate alone tell us little about a person's context: your heartbeat could be increased due to experiencing a negative, but also positive emotion. For example, you could be afraid or excited, stressed or sexually aroused.'
'We call wearable data in my field 'objective' data-but I'm not sure that's the right label.' - Eiko Fried
Important questions
'The findings raise important questions about what wearable data can or can't tell us about mental states', says Fried. 'Not only is this relevant for individuals who rely on information provided by smartwatches to guide their life habits. It's also important for clinical and social sciences, given the common belief among researchers that wearables can replace traditional self-reports to reduce research participant burden.'
'We call wearable data in my field 'objective' data-but I'm not sure that's the right label. Wearable data are a crucial data source that offer important insights, but there's no such thing as objective data, given challenges like measurement error or missing data. In our case, the wearable stress data is clearly not an objective measure of what we perceive as stress. Augmenting wearable data with self-report has the potential to give us a much improved holistic insight into a person's thoughts, feelings, and behaviors.'
Siepe's and Fried's paper 'Associations between ecological momentary assessment and passive sensor data in a large student sample' has been accepted in the Journal of Psychopathology and Clinical Science, and will be published soon.
