Artificial intelligence (AI) systems that companies claim to be able to “read” facial expressions are based on outdated science and risk being unreliable and discriminatory, has one of the world's foremost experts in the field of emotion psychology warned.
Lisa Feldman Barrett, professor of psychology at Northeastern University, said that such technologies seem to be ignoring a growing body of evidence that undermines the idea that basic facial expressions are universal in all cultures. As a result, such technologies – some of which are already being used in practice – run the risk of being unreliable or discriminatory, she said.
“I don't know how companies can continue to justify what they do if it is really clear what the evidence is,” she said. “There are some companies that just keep claiming things that can't possibly be true.”
Her warning comes when such systems are rolled out for a growing number of applications. In October, Unilever claimed that it had saved 100,000 hours in recruiting staff last year by using such software for analyzing video interviews.
The AI system, developed by the HireVue company, scans the facial expressions, body language and word choice of candidates and compares them with characteristics that are considered to be related to the success of a job.
Amazon claims that its own face recognition system, Rekognition, can detect seven basic emotions – happiness, sadness, anger, surprise, disgust, calmness and confusion. The EU is reportedly testing software that is said to be able to detect fraud through an analysis of micro-expressions in an effort to strengthen border security.
“Based on the published scientific evidence, our opinion is that [these technologies] should not be deployed and used to make subsequent decisions about people's lives,” Feldman Barrett said.
AI can read your emotions. Is that necessary?
Prior to a speech at the annual meeting of the American Association for the Advancement of Science in Seattle, Feldman Barrett said that the idea of universal facial expressions for happiness, sadness, fear, anger, surprise, and disgust in the 1960s had power won after an American psychologist, Paul Ekman, did research in Papua New Guinea, showing that members of an isolated tribe gave similar answers to Americans when asked to match photos of people with different facial expressions with different ones scenarios, such as “the Bobby dog died”.
However, there is increasing evidence that in addition to these basic stereotypes there is a big difference in the way people express their emotions, both within and outside cultures.
In Western cultures, for example, people seem to frown only about 30% of the time when they are angry, she said, meaning that they move their faces differently about 70% of the time.
“There is low reliability,” said Feldman Barrett. “And people often frown when they're not angry. That's what we would call low specificity. People frown when they concentrate very hard, when you tell a bad joke, when they have gas.”
The expression that is supposed to be universal for fear is the supposed stereotype for a threat or anger face in Malaysia, she said. There are also major differences within cultures in the way people express emotions, while context such as body language and with whom a person talks is critical.
“AI is largely trained on the assumption that everyone expresses emotions in the same way,” she said. “Very powerful technology is used to answer very simplistic questions. ”