A new study in Denmark used machine learning techniques on photographs of faces of Danish politicians to predict whether their political ideology is left- or right-wing. The accuracy of predictions was 61%. Faces of right-wing politicians were more likely to have happy and less likely to have neutral facial expressions. Women with attractive faces were more likely to be right-wing, while women whose faces showed contempt were more likely to be left-wing. The study was published in Scientific Reports.
The human face is highly expressive. It uses a complex network of muscles for various functions such as facial expressions, speaking, chewing, and eye movements. There are more than 40 individual muscles in the face, making it the region with the highest concentration of muscles. These muscles allow us to convey a wide range of emotions and perform intricate movements that are essential for communication and daily activities.
Humans infer a wide variety of information about other people based on their faces. These includes judgements about personality, intelligence, political ideology, sexual orientation and many other psychological and social characteristics. However, while humans make these inferences almost automatically in their daily lives, it remains contentious which exactly characteristics of faces are used to make these inferences and how.
Study author Stig Hebbelstrup and his colleagues wanted to explore whether it is possible to use computational neural networks to predict political ideology from a single facial photograph. Computational neural networks are a class of algorithms inspired by the structure and function of biological brains. They consist of interconnected nodes, called artificial neurons or units, organized into layers. Each neuron takes input from the previous layer, applies a function, and passes the output to the next layer.
The primary purpose of computational neural networks is to learn patterns and relationships within data by adjusting the connections between neurons. This learning process, often referred to as training or optimization, is typically achieved using a technique called backpropagation. This means that after an error is made in the outcome, changes are applied to the functions in preceding nodes in order to correct it.
To train this neural network, researchers used a set of publicly available photos of political candidates from the 2017 Danish Municipal elections. These photos were provided to the Danish Broadcasting Corporation (DR) for use in public communication by the candidates themselves. The authors note that these elections took place in a non-polarized setting. The candidates have not been highly selected through competitive elections within their parties and are thus referred to as the “last amateurs in politics” by Danish political scientists.
The initial dataset consisted of 5,230 facial photographs. However, the researchers excluded photos of candidates representing parties with less-defined ideologies, that could not be classified as left- or right-wing, photos of faces that were inadequate for machine processing, and those that were not in color.
An author who did not know the names or parties of the candidate then manually inspected the photos and excluded photos of candidates who did not appear to be of European ethnic origin. The reason for this exclusion was that candidates of non-European origin, while easy to visually identify, were 2.5 times more likely to be representing left-wing parties.
Finally, the authors excluded photos of candidates with beards, noting that beards can impair the detection of facial expressions and some other analyses as well. The algorithm was separately trained on male and female photos. The final dataset consisted of 4647 photos, of which 1442 were female.
Hebbelstrup and his colleagues tested the accuracy of the algorithm on an additional sample of Danish parliamentarians. This sample was divided into males and females with algorithm applied separately to each, but no other exclusions were done. All photos were edited to ensure that they only show the faces and to exclude any other elements that could be used to infer ideology (such as background colors or clothes).
The researchers created measures of the emotional state expressed by the face using the Face API from Microsoft’s Azure’s Cognitive Services. Results showed that 80% of faces showed happiness, while 19% had neutral expressions. The authors attribute this to the algorithm being unable to accurately identify other types of facial expressions. Additionally, they used algorithms to assess the attractiveness of candidates and the masculinity of male candidates.
Results showed that the neural network trained on these data was 61% accurate in predicting ideology based on a facial photograph in both males and females. In other words, the accuracy of the prediction algorithm is better than chance.
Analysis of the facial characteristics that were crucial in making decisions about ideology revealed that masculinity and attractiveness were not linked to ideology in males. However, more attractive females were more likely to be representatives of right-wing parties. Happy faces, both male and female, were also more likely to be representatives of right-wing parties, while faces with neutral expressions were more likely to be left-wing. Although rare, women whose faces showed contempt were more likely to be representatives of left-wing parties.
“Our results confirmed the threat to privacy posed by deep learning approaches. Using a pre-developed and readily available network that was trained and validated exclusively on publicly available data, we were able to predict the ideology of the pictured person roughly 60% of the time in two samples,” the researchers conclude.
“We also provide the first demonstration that model-predicted ideology connects to independently classifiable features of the face. For females (though not males), high attractiveness scores were found among those the model identified as likely to be conservative. These results are credible given that previous research using human raters has also highlighted a link between attractiveness and conservatism.”
The study makes a valuable contribution to scientific understanding of the links between ideologies and appearance. However, it should be noted that it also has limitations that need to be taken into account. Notably, authors do not provide percentages of right-wing and left-wing politicians in the sample, but use a flip-coin chance as reference.
However, if one of these categories of politicians constitutes more than 60% of the sample of photographs than simply classifying all candidates as belonging to the dominant category would produce greater accuracy than that achieved by the study algorithm. This would put the results in a different light. Additionally, all politicians whose photographs were included in the study were Danish. It is possible that results on other populations would not be the same.
The study, “Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra‑facial information”, was authored by Stig Hebbelstrup, Rye Rasmussen, Steven G. Ludeke, and Robert Klemmensen.
