Are you happy or angry in front of the screen?

In your work day, when you’re in front of the screen do you mostly feel joy, anger or sadness? Are there tasks at work that cause you a lot of frustration? Which tasks bring you the most joy? Do interactions with your colleagues tend to make you feel confident or angry?

This is the kind of information you can get from a new piece of Canadian-made, French-language facial-recognition technology called EmoScienS that helps you track your emotions and then tap into tools to better regulate them.

The Montreal start-up that developed the technology was launched two years ago by Pierrich Plusquellec, an associate professor at the School of Psychoeducation of Université de Montréal, with his colleagues Nathe François, a specialist in knowledge transfer, and Ted Hill, a specialist in automatic emotion recognition.

Their organization's mission is to enable people to use data from their emotional facial expressions to improve their well-being.

“People don't ask themselves many questions about what they're going through in life, unless they're a Buddhist monk,” said Plusquellec, a behavioral biologist whose work focuses on nonverbal indicators of stress.

But knowing what emotions you’re experiencing can make a big difference, he added.

“Research suggests that emotional awareness is a cure for chronic stress, anxiety, rumination and depression. I wanted to use technology to mirror people’s emotions – something like the Fitbit, but on an emotional level. Because once you become aware of your emotions, you can test different ways to regulate them.”

Improving well-being in the workplace

EmoScienS uses artificial intelligence – specifically, automatic emotion recognition – to get its data. After initial support from German-based Center for Nanotechnology (CeNTtech), EmoScienS is now being supported by Montreal venture capital fund Esplanade, a catalyst for social and environmental-impact entrepreneurs. The commercialization phase has begun, targeting organizations of at least 50 employees.

“Thanks to EmoScienS, companies can see the emotional state not of individuals, because the data is anonymized, but of teams," explained Plusquellec. “With real-time monitoring, the company's leader can quickly know if something is going on in a department and act accordingly.”

With EmoScienS, employees decide whether they want to share their data anonymously with their employer. The benefit to them, in addition to allowing their manager to better monitor team dynamics, is having access to their individual data via a secure dashboard. “They can see which moments in the day they were particularly happy, sad or angry and make the link with what they were specifically doing as a task or activity,” said Plusquellec.

Take anger, for example. “It has a clear function, which is to indicate the presence of an obstacle to a value or a goal,” says Plusquellec. “By looking at the peaks of anger in a day, we are able to identify the obstacles and therefore to know when they will occur again. This way, we can plan a strategy to regulate our anger at that time.”

In his work as a researcher, Plusquellec has found that many people have a huge deficit in how they regulate their emotions. To help, EmoScienS brings what he’s learned through science to the end users.

“We propose different techniques, such as that of Barbara Fredrickson, which consists of  cultivating positive emotions in one's daily life,” he said. “Or James Gross' techniques for cognitive restructuring when you get angry. Or, you can try something else entirely, like doing yoga to see if it decreases your negative emotions. "

He also points out that psychologist and economist Daniel Kahneman, who won the Nobel Prize in economics in 2002, considers one of the neglected dimensions of well-being to be the balance between positive and negative emotions experienced in everyday life.

Managing data responsibly

EmoScienS' activities fall into the broad category of facial recognition technologies. Privacy laws require companies that collect these types of data to obtain consent from the individuals involved.

“But we went further, because ethics and social responsibility are at the heart of our concerns," said Plusquellec. “A technology always has a dark side and a light side, and we wanted to keep it on the light side.”

EmoScienS was funded in 2020 by the Partenariat-UdeM Program and works with the CHUM, Université de Montréal’s teaching hospital, a signatory of the Montreal Declaration for Responsible AI, another UdeM initiative.

"We are also collaborating with researchers at HEC Montreal” – UdeM’s affiliated business school – “to have our clients sign a social responsibility charter in which they commit to not using data to try to find an individual, for example. It's important to us that this is very well framed.”

In association with the mental-health Association québécoise des parents et amis de la personne atteinte de maladie mentale (AQPAMM), EmoScienS also obtained funding this year from Quebec’s Ministry of Economy and Innovation. The goal is to train leaders and managers on mental-health issues and to measure the impact of this preventive approach using EmoScienS' technology.

Concluded the professor: “Mental health is particularly topical these days with COVID-19 causing people to spend even more time on the screen, so we think our tool can be useful to them.”