Study finds emotion recognition goes beyond facial expressions

A person’s facial expressions provide important information for us to recognize their emotions. But there’s more to this process than that. This is according to a study conducted by Dr Leda Berio and Professor Albert Nguyen of the Second Institute of Philosophy at the Ruhr University Bochum in Germany.

The research team describes emotion recognition not as a separate module, but as part of a comprehensive process that helps form overall impressions of others. This process of forming an impression of a person includes not only background information, but also physical and cultural characteristics. This paper was published in the journal Philosophy and Phenomenological Research on September 24, 2024.

Understanding the situation influences how emotions are recognized

In the 1970s, a theory was proposed that the face is a window to our emotions. Researcher Paul Ekman described basic emotions such as fear, anger, disgust, joy, and sadness using typical facial expressions that were found to be similar across cultures.

“However, in recent years it has become increasingly clear that there are many situations in life in which typical facial expressions are not necessarily important information when assessing the emotions of others.” Nguyen points out.

“People almost universally rate a typical expression of fear as anger, but with the background knowledge that the person being evaluated had clearly reserved a table and was turned away by the waiter. .”

In such situations, a person expects the person to be angry, and this expectation determines the perception of the person’s emotion, even though the person’s facial expression is usually due to another emotion. .

“Furthermore, we can sometimes recognize emotions without seeing faces. For example, the fear experienced by a person being attacked by a growling dog can be expressed by a frozen or frightened posture. “Even though I only see it from behind.” Berio.

Emotional recognition is part of a person’s overall impression

Berio and Nguyen propose that emotion recognition is a subprocess of a person’s ability to form an overall impression. By doing so, people are guided by certain characteristics of the other person. Examples include physical characteristics such as skin color, age, and gender, cultural characteristics such as clothing and attractiveness, and situational characteristics such as facial expressions, gestures, and posture.

Based on these characteristics, people tend to quickly evaluate others and quickly associate them with social status or certain personality traits. These associations determine how we perceive other people’s emotions.

“If we perceive the other person as a woman and she exhibits a negative emotion, we are more likely to judge that emotion to be fear, whereas for men it may be interpreted as anger. ,” notes Berio.

Background information is also included in the evaluation

In addition to feature recognition and initial associations, we also maintain detailed portraits of individuals in our social circles, such as family, friends, and colleagues, that we use as background information.

“When someone in our family suffers from Parkinson’s disease, we learn to evaluate that person’s typical angry facial expressions as neutral, because hard facial expressions are part of Parkinson’s disease. Because we recognize that,” Berio said.

Background information also includes character models of typical occupational groups. “We have stereotypes about the social roles and responsibilities of doctors, students, and workers, for example,” Nguyen says. “For example, we generally perceive doctors to be less emotional, but this changes how we assess their emotions.”

In other words, people draw on a rich set of traits and background knowledge to assess the emotions of others. In rare cases, we can read people’s emotions just from their facial expressions.

“All of this has implications for emotion recognition using artificial intelligence (AI), which becomes a reliable option only if the AI ​​does not rely solely on facial expressions, which most systems currently do. ” says Newen.