Submitted by Rachel Gardner on Fri, 24/09/2021 - 15:25
Developing smartphone technology that can sense our emotions and tell if we are happy or sad has just won researchers here a 10-Year Impact Award.
The accolade was presented at UbiComp 2021, the Pervasive and Ubiquitous Computing Conference.
It was given for the long-term impact of the 2010 research paper EmotionSense: A Mobile Phones-based Adaptive Platform for Experimental Social Psychology Research.
EmotionSense was a project in which computer scientists and psychologists proved they could develop an effective system for tracking individuals’ emotional behaviour via ordinary, off-the-shelf mobile phones using speech-recognition software and phone sensors.
The paper was authored by three authors in this Department (Kiran K Rachuri, Cecilia Mascolo and Andrius Aucinas). They were working in collaboration with Mirco Musolesi (then at the University of St Andrews), Peter Jason Rentfrow (in the Cambridge Department of Psychology) and Chris Longworth (then in the Cambridge Department of Engineering).
EmotionSense was designed to use a phone's recording tools to analyse audio samples of the user speaking and compare them with an existing speech library. The built-in global positioning system software was used to cross-reference the audio samples with the user's location, and Bluetooth technology and the phone's recording data were used to identify who the user was talking to.
The aim was to use the data to enable psychologists to better understand how people's emotions are influenced by their surroundings, the time of day, or their relationships with others.
It attracted a lot of interest, garnering headlines such as Cambridge scientists prove 'home is happiest' with help from mobile phones (Telegraph, September 2010) and The mobile phone technology that can tell if you're happy or sad (Daily Mail, September 2010).
"Everyone has a mobile phone, so potentially they are a perfect tool if you want to track the behaviour or emotional condition of large numbers of people," Cecilia (now Professor) Mascolo said at the time. "What we are trying to produce is a completely non-intrusive means of achieving that which also respects privacy."
In their award-winning 2010 paper, the researchers describe how the EmotionSense system was implemented on a Nokia 6210 Navigator phone and how they ran a social psychology experiment to evaluate its usefulness in which they compared the data extracted via EmotionSense with the data provided by participants via more traditional questionnaires.
The study showed that the information collected by EmotionSense could be used by social scientists in order to understand the patterns of interaction and the correlation of emotions with places, groups, and activity.
It also showed that phone users’ emotions could be automatically detected when using the technology on off-the-shelf mobile phones, and that speaking and interactions could be correlated with activity and location measures to see how people’s emotions were influenced by their surroundings.
Congratulations to all the authors on this award.
