Submitted by Rachel Gardner on Mon, 25/11/2024 - 15:08
Exploring how to turn 'hearable' devices like earbuds into genuinely trustworthy and reliable instruments for measuring human health and fitness: that's the target of a new research project that will shortly get underway.
The project 'HearFit: Hearable Sensing Systems and Machine Learning for Health and Fitness' will be funded by the EPSRC Open Fellowship that has just been awarded to Prof Cecilia Mascolo (right). Many congratulations to her!
These sought-after Fellowships support talented and ambitious researchers in pursuing high-level research and developing their research careers over a five-year period.
Prof Mascolo hopes it will help her lay the foundations for the next generation of affordable and efficient sensing devices and machine learning modelling for monitoring wearers' activity, fitness and health over time.
Having sensors that don't just take a snapshot of our state of health, but collect health information from us over extended periods of time, would be very valuable. It could really impact the way we study an individual's health.
Prof Cecilia Mascolo
Cecilia is Professor of Mobile Systems at the University of Cambridge. She also co-directs the Centre for Mobile, Wearable Systems and Augmented Intelligence here, which focuses on next-generation mobile and wearable technology, as well as mobile applications.
She is a pioneer in devising frameworks to collect sensing data from devices such as phones and wearables with the purpose of developing models to understand behaviour and health. During the pandemic, she and her colleagues developed the COVID-19 Sounds App which collects and analyses short recordings of users coughing and breathing to detect if they are suffering from COVID-19.
Since then, she has been working on ways to turn the devices we wear – such as earbuds – into mobile monitors that can collect data about our state of health, and developing cutting edge machine learning tools to evaluate that data on the device itself.
Work by Cecilia with researchers Kayla-Jade Butkow (right) and Yang Liu showed for the first time that heart and respiration rates could be detected by earbuds' in-ear microphones while the wearers were carrying out a range of activities, including running.
"Imagine a future where your earbuds do more than play music – they become your personal health companion," Kayla and Yang say. "Whether you're asleep, in a remote meeting, working, walking or running, our innovative technology enables seamless monitoring of your heart and breathing rates."
Such capabilities will address significant challenges faced by today's wellness monitoring solutions. A smartwatch, for example, can measure a wearer's heart rate at rest but its accuracy often falters when they're moving because that movement disrupts signal quality. The advantage of in-ear devices is that they don't move so much.
Because of this, "our technology elevates earphones into advanced wellness monitoring tools, leveraging their rich sensing capabilities, reduced motion interference and close proximity to key arteries," add Kayla and Yang. "This combination enables more robust, precise and consistent monitoring, unlocking a new level of insight into wearers' health and well-being. Beyond monitoring these two vital signs, we see huge potential for earbuds to track your overall health and well-being as you go about your daily activities."
Imagine a future where your earbuds do more than play music – they become your personal health companion. Whether you're asleep, in a remote meeting, working, walking or running, our innovative technology enables seamless monitoring of your heart and breathing rates.
Kayla-Jade Butkow and Yang Liu
Using these devices as health sensors would be very useful, thy says, as they are so widely worn nowadays. They are also ultra light and portable, and much more affordable than standard clinical diagnostic devices.
Using everyday devices for health monitoring
"These devices are everywhere," add Kayla and Yang (left), "generating a lot of data about our behaviour and how well our bodies are functioning. This offers us an unprecedented – and detailed – window onto human wellbeing and fitness. They also have the potential to be used at scale to measure and monitor the health of a population over time."
So-called 'hearables' have become much more widely worn in the last few years, being used for everything from enjoying music to listening to the commands of a fitness app during exercise. Pandemic lockdowns encouraged their use still further as workers plugged them into their ears while attending virtual meetings from home. And now companies are starting to market them as noise-cancelling devices that improve wearers' sleep.
"While we don't know what the next generation devices will look like in a few years, we're certain the foundations we develop in this project will be applicable to them to understand human well being, fitness and health.”
The project aims to improve the methodology by which we process and analyse the data to ensure that that we can cope with limited precisions of the sensors and the limited data. Also, while the devices allow data from the wearer to be collected over time, what is key is that the learning models can make sense of that data when they only have limited information about the wearer’s behaviour to train on.
Enhancing users' privacy
Another of Cecilia's goals is to have the data collected by the hearables analysed and evaluated on the device itself as much as possible. "This," she says, "would ensure much more privacy for users than if – as at present – the data is transmitted to commercial servers for analysis. However with devices with such small form factors, this is a challenging task."
The overall aim of this fellowship is to lay the foundations for hearable computing to become an open reliable, trustable and robust means to understand our behaviour, fitness and health.
"Having sensors that don't just take a snapshot of our state of health, but collect health information from us over extended periods of time, would be very valuable," Cecilia says. "It could really impact the way we study an individual's health and the onset and progression – and their recovery from – clinical disease.
"And this could happen on a broad scale. The technology could be scaled up to help us measure the health of large populations and monitor the progression of future pandemics."
