AI & Behavioral Science: A Toxic Cocktail

AI. The two letters that strike fear and awe in a curious public searching for an inexplicable, unmanageable, and irreversible problem. Now what? AI can look at our faces and determine our political orientation? Of course, we might be worried about the obvious political thrashing we take part in on social media, but that wouldn’t be as much fun. Where’s the fear in that? It would be like realizing we mistakenly got on the kid’s train ride instead of the roller coaster at the amusement park. So, let’s go with the AI fear scenario. We can mix politics, AI, and fear all in the same discussion, a gasoline/kerosene cocktail with a lit match chaser.

The Facial Recognition Political Orientation Study

The study we’ll review is titled, “Facial Recognition Technology and Human Raters Can Predict Political Orientation From Images of Expressionless Faces Even When Controlling for Demographics and Self-Presentation.”¹ The Stanford University institutional review board approved the study.²

The study’s hypothesis, which can be derived from its name, is that someone’s facial features can reveal their political orientation, liberal or conservative. If you don’t think that sounds weird enough, after reviewing the study we will mention the Federal Government’s current AI intervention strategy.

Facial Recognition Fear

The study presents interesting ideas. As someone who received a Sociology degree in the early ‘70s, it was a return to the past of reading over a detailed research paper that utilized documented information from previous studies. For people who are not into behavioral science, the actual study could be a tedious read. We’ll take the tedium out of it, and highlight what will have the greatest impact on us now and in the future.

AI Phobia

I believe most of the pseudo-news media and social media never made it past the single paragraph Public Significance Statement in the study. This is where the authors state:

“This indicates a connection between political leanings and inherent facial characteristics, which are largely beyond an individual’s control. Our findings underscore the urgency for scholars, the public, and policymakers to recognize and address the potential risks of facial recognition technology to personal privacy.”³

I figured that was my action call to write a post. I may not be a scholar, and I’m not part of any policy making apparatus, but I am in the public when I go to Walmart. So, let’s take a swing at it.

Is it Relevant?

In any research study you start with a hypothesis. As I mentioned in the intro, the hypothesis is the same as the name of the study. Then you measure the relationship between two variables in the study. It appears the two variables are: 1) Predicted Political Orientation, and 2) Actual Political Orientation.

The tool to measure the relationship between a research study’s variables is the “Pearson Correlation Coefficient.” The Pearson correlation coefficient measures the accuracy of the correlation. The values range from a negative 1 to a positive 1. The closer the value is to a positive 1, the greater the probability of a positive correlation between the two variables. The closer the value is to negative 1 the greater the probability of an inverse correlation between the two variables.⁴

I may be out of my league here, but the Pearson correlation coefficients, (r = .22) for the AI algorithm and (r = .21) for humans, do not suggest the human raters or the Facial Recognition algorithm demonstrated an unusually high level of accuracy in guessing political orientation. Also, the humans appeared to be almost as good as the Facial Recognition algorithm in determining political orientation. Maybe we should find out more about these humans who are capable of mimicking Artificial Intelligence.

The study made a further observation.

“The algorithm’s predictive accuracy was even higher (r = .31) when it leveraged information on participants’ age, gender, and ethnicity.”⁵

So, now the AI algorithm has a person’s photo, it knows their age, gender, and ethnicity, and then determines with a low degree of moderate accuracy, (r = .31), the political orientation of the person. Isn’t that what we mundane individuals call voting blocs? I’ll turn that one over to pollsters, think tanks, and the media. They can barrage us with intellectualism on the topic since they are society’s credentialed literati.

In their subsequent “FAQ” addition to the study, the authors addressed the low variable correlation. They posed the question to themselves, “Correlation of r=.2 seems very small, doesn’t it?”⁶ They stated why they believe the correlation is “not small at all,” and delineated the reasoning for their assertion. That is their educated opinion.

I agree there is a correlation, but I do not think it is significant. That is my opinion, and it is not meant to disparage the study or the researchers. The study sheds light on a critical area of Artificial Intelligence.

Findings

In the “Discussion” section of the study, the researchers referenced what they mentioned in the Abstract and Public Significance Statement.

“The algorithm studied here, with a prediction accuracy of r = .22, does not allow conclusively determining one’s political views, in the same way as job interviews, with a predictive accuracy of r = .20, cannot conclusively determine future job performance. Nevertheless, even moderately accurate algorithms can have a tremendous impact when applied to large populations in high-stakes contexts.”⁷

This is a less alarmist perspective than suggested in the Public Significance Statement. The problem is the media, politicians, and much of the public prefer an alarmist, fear-laden perspective. After all, very few people will click on a story with a headline that states, “AI is just a bit better than humans at guessing your political orientation.”

The researchers made an interesting observation in the Discussion section regarding lower facial characteristics.

“Among the interpretable facial features examined here, only lower face size was clearly associated with political orientation (liberals tended to have smaller lower faces); consistent with previous findings, fWHR showed no predictive power (Kosinski, 2017).”⁸

In the Abstract section the researchers stated:

“The analysis of facial features associated with political orientation revealed that conservatives tended to have larger lower faces.”⁹

If you see someone who isn’t your dentist staring at your lower face, you have a problem that requires immediately releasing the contents of your tear gas canister, or they think they can guess your political orientation. If it’s latter, just tell them that there are some people who are so naïve, they think they can guess your political ideology from the shape of your lower face.

The Lack of Magic Behind It

We’ll break this into two parts. First, we’ll address facial recognition and the algorithm used in the researchers’ study. Second, we’ll summarize AI to take some of awe, mystery, and fear out of it. It will be brief, but relevant to our discussion.

Facial Recognition

The collecting of facial images has been going on for a while. A cursory look at it reveals the Labelled Faces in the Wild (LFW) dataset came about in 2007.¹⁰ One of my favorites, only because I like the name, is the MegaFace dataset from 2016 with 4.7 million images.¹¹ Microsoft got into the program with its Ms-Celeb-1M dataset in 2016 with 10 million images from 100,000 celebrities.¹² Society’s current panic mode is based on the VGGFace2 dataset¹³ and model.

Training the algorithm is an important part of the study. The only information on algorithm training that I found stated they used a “a commonly used facial recognition algorithm.” Perhaps there’s more, but I did not see it. This is part of the introduction to “Study 1” which describes the AI algorithmic “rater.”

“We employ a commonly used facial recognition algorithm, VGGFace2, in ResNet-50-256D architecture (Cao et al., 2018). The neural network underlying this model was trained on 9,131 people and their 3.3 million facial images, varying in pose, facial expression, illumination, and other superficial characteristics. It was trained to convert diverse facial images of an individual person into face descriptors, or a numerical vector that is both unique to that individual and consistent across their different images.”¹⁴

So, facial image collection has been going on for at least 17 years. This does not include forcing our iPhones to recognize our face before we interact with it. I keep trying to train mine with a photo of Dwayne “The Rock” Johnson. Somehow my phone is not smart enough to see the resemblance.

Artificial Intelligence

Many people attribute the advent of Artificial Intelligence to a computer scientist, John McCarthy, for the work he did during the mid-1950s.¹⁵ McCarthy also developed the LISP (List Processing) Programming Language, “which for decades was the principal language for AI work in the United States, before it was supplanted in the 21st century by such languages as Python, Java, and C++.”¹⁶

There were others before McCarthy who pioneered neural networks. A neural network is “a computer program that operates in a manner inspired by the natural neural network in the brain.”¹⁷ An interesting Britannica article gives us insight:

“The theoretical basis of neural networks was developed in 1943 by the neurophysiologist Warren McCulloch of the University of Illinois and the mathematician Walter Pitts of the University of Chicago. In 1954 Belmont Farley and Wesley Clark of the Massachusetts Institute of Technology succeeded in running the first simple neural network.”¹⁸

Artificial Intelligence + Facial Recognition = Fear

Facial recognition databases have been with us since at least 2007. Artificial Intelligence and its predecessor, neural networks, started in the 1950s. We had to wait until 2024 to give us what we needed, and wanted, to make it relevant to the average person. Politics and fear. That is current society’s contribution to the discussion.

Government Response to Artificial Intelligence

I understand the researchers believe there is an imminent threat from AI facial recognition technology. Their platform and credentials allow them to have that opinion. A call to legislative action should not be based solely on this research. The House of Representatives already formed an AI Task Force in February of 2024¹⁹, and “Colorado U.S. Senator Michael Bennet introduced the Assuring Safe, Secure, Ethical, and Stable Systems for AI (ASSESS AI) Act to make the U.S. government lead by example in the responsible use of artificial intelligence (AI).”²⁰

Mike Johnson, the current Speaker of the House, appointed Republican Representative Jay Obernolte to chair the House AI Task Force Team in addition to 11 other representatives. Minority House Leader Hakeem Jeffries appointed Democratic Representative Ted Lieu as co-chair of the AI Task Force and an additional 11 members.²¹ The chairman, Obernolte, and co-chairman, Lieu, have impressive academic credentials.

Jay Obernolte

“In addition to owning a videogame development studio, Rep. Obernolte is a certified flight instructor and a teacher of martial arts. He holds a B.S. in Engineering and Applied Science from the California Institute of Technology (Caltech), an M.S. in Artificial Intelligence from the University of California, Los Angeles (UCLA), and a Doctorate in Public Administration from California Baptist University (CBU).”²²

Ted Lieu

“Ted would go on to attend Stanford for his undergraduate degrees in Computer Science and Political Science, and then Georgetown University, where he received his law degree magna cum laude after serving as Editor-in-Chief of the law review. Ted also received four American Jurisprudence Awards.”²³

“As one of four Members of Congress with a computer science degree, Ted is frequently sought out for his insights on technology and innovation matters including artificial intelligence, cybersecurity and innovation as well as the creative economy.”²⁴

We, the Public

So, should we, the public, feel reassured that Congressional oversight will address not only our privacy concerns, but also any other threat posed by Artificial Intelligence? First, out of the 435 voting members of the House of Representatives and 100 members of the Senate, there are only four who have Computer Science degrees? This is 2024, not 1984, although it may sometimes feel like it. A cynic might think Congress is just a bunch of lawyers and career politicians mixed in with a bartender and one or two baristas.

Second, the House “AI Task Force” is composed of a total of 24 members. Some of us who worked in the Technology sector for decades understand one important rule. When you get too many people in the room to address a technology issue, it devolves into chaos control rather than problem resolution. Maybe it takes 24 people to test out the latest AI video game.

The third issue is the most important. The Facial Recognition study is about the use of Artificial Intelligence in behavioral science. I did not see that concern mentioned as a priority or even an interest for the AI Task Force. Manipulating humans’ perceptions, thoughts, attitudes, and actions is the purview of behavioral science. Those concerns have universal ramifications affecting current political and social dynamics, and for all future generations.

Could someone use AI to determine people’s perceptions, thoughts, and attitudes, then predict their actions through a simple exchange of information on social media or a credit application? Could someone use AI to create scenarios that influence or alter humans’ perceptions, thoughts, attitudes, and actions? We are already besieged with targeted advertising when we use the internet. That is not accidental. The looming question is, would a foreign adversary be interested in influencing our political and social landscape utilizing a combination of behavioral science and AI?

I did not research each member on the AI Task Force. Apparently, few have the background to evaluate the future implications of Artificial Intelligence in general.²⁵ In my brief research on the “AI Task Force,” I did not find any reference to AI in relation to behavioral science, or anyone credentialed in the field. I will admit when viewing Obernolte’s and Lieu’s photographs, I found myself scrutinizing their lower faces.

Conclusion

This study drew my interest because it tapped into two of my favorite subject areas. Behavioral Science and computer technology. My first degree is in Sociology. I had the privilege of taking classes from a dedicated researcher in the field of Behavioral Science. She made the subject matter relevant. My second degree is in Computer Science, Math, and Systems Design. I am also grateful to one of my toughest instructors, whose disciplined approach to technology demanded precision. She made the subject matter concrete. One of my Computer Science courses was in the LISP programming language, which served as my introduction to Artificial Intelligence in the fall of 1980.

The 696-word article announcing the formation of the AI Task Force used the word “bipartisan” 12 times.²⁶ A bipartisan group of politicians led by computer science “experts” is meant to reassure the public that legislation will be in everyone’s best interest.

Democratic Leader Hakeem Jeffries, who appointed Ted Lieu co-chair the AI Task Force, wants to ensure “everyday Americans benefit from these advancements in an equitable manner.”²⁷ In an interview, Representative Obernolte expressed it is the responsibility of Congress “to make it clear what the guardrails are on what the states are allowed to do in AI, and what is reserved for the Federal Government as a matter of interstate commerce.”²⁸ Are these pressing issues in AI? As Ronald Regan said, “I think you all know that I’ve always felt the nine most terrifying words in the English language are: I’m from the Government, and I’m here to help.”²⁹

Instead of a bipartisan group of politicians, why not try a nonpartisan group of experts in their respective fields? My former computer science and sociology instructors would have been perfect choices to lead or at least be a part of that committee. Kosinski and the other researchers who did the Facial Recognition study would be natural additions since they have the background and have shown interest in the issue. They, along with other nonpartisan experts in their respective fields, would add meaningful insight to the conversation. Politicians exploit the conversation. It’s their job.

The glaring omission of behavioral science in addressing AI issues should concern us. Misuse of AI in the realm of behavioral science is the most probable scenario now and in the future. Humans use technology. Artificial Intelligence enables technology to use humans.

---

1. Michal Kosinski, Poruz Khambatta, and Yilun Wang, “Facial Recognition Technology and Human Raters Can Predict Political Orientation From Images of Expressionless Faces Even When Controlling for Demographics and Self-Presentation,” American Psychologist, Online First Publication, February 19, 2024, https://psycnet.apa.org/fulltext/2024-65164-001.html#articleRefList_amp0001295. ↩︎
2. Kosinski et al., “Facial Recognition Technology,” “Public Significance Statement.” ↩︎
3. Kosinski et al., “Facial Recognition Technology,” “Public Significance Statement.” ↩︎
4. Shaun Turney, “Pearson Correlation Coefficient ® | Guide & Examples,” Scribbr, May 13, 2022, revised February 10, 2024, accessed May 6, 2024, https://www.scribbr.com/statistics/pearson-correlation-coefficient/. ↩︎
5. Kosinski et al., “Facial Recognition Technology,” “Abstract.” ↩︎
6. Kosinski et al., “Facial Recognition Technology,” “FAQ,” https://osf.io/nuz2m/wiki/home/. ↩︎
7. Kosinski et al., “Facial Recognition Technology,” “Discussion.” ↩︎
8. Kosinski et al., “Facial Recognition Technology,” “Discussion.” ↩︎
9. Kosinski et al., “Facial Recognition Technology,” “Abstract.” ↩︎
10. Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman, “VGGFace2: A dataset for recognising faces across pose and age,” ar5iv, accessed May 6, 2024, https://ar5iv.labs.arxiv.org/html/1710.08092 sec. II Dataset Review. ↩︎
11. Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman, “VGGFace2: A dataset,” sec. II Dataset Review. ↩︎
12. Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman, “VGGFace2: A dataset,” sec. II Dataset Review. ↩︎
13. Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman, “VGGFace2: A dataset for recognising faces across pose and age,” ar5iv, accessed May 6, 2024, https://ar5iv.labs.arxiv.org/html/1710.08092 ↩︎
14. Kosinski et al., “Facial Recognition Technology,” “Study 1: Standardized Images of Neutral Faces.” ↩︎
15. “Professor John McCarthy,” Stanford University, accessed May 6, 2024, http://jmc.stanford.edu/index.html. ↩︎
16. Copeland, B. “artificial intelligence.” Encyclopedia Britannica, May 6, 2024. https://www.britannica.com/technology/artificial-intelligence. ↩︎
17. Zwass, V. “neural network.” Encyclopedia Britannica, April 25, 2024. https://www.britannica.com/technology/neural-network. ↩︎
18. Zwass, V. “neural network.” ↩︎
19. “House Launches Bipartisan Task Force on Artificial Intelligence,” Congressman Ted Lieu, February 20, 2024, accessed May 6, 2024, https://lieu.house.gov/media-center/press-releases/house-launches-bipartisan-task-force-artificial-intelligence. ↩︎
20. “Bennet Introduces Legislation to Stand Up An AI Task Force to Ensure Responsible Use of The Technology By The Federal Government,” U.S. Senator Michael Bennet, April 28, 2023, accessed May 6, 2024, https://www.bennet.senate.gov/public/index.cfm/2023/4/bennet-introduces-legislation-to-stand-up-an-ai-task-force-to-ensure-responsible-use-of-the-technology-by-the-federal-government. ↩︎
21. Lieu, “House Launches Bipartisan Task Force.” ↩︎
22. “About Representative Jay Obernolte,” U.S. Congressman Jay Obernolte, accessed May 6, 2024, https://obernolte.house.gov/about. ↩︎
23. “Meet Ted,” Congressman Ted Lieu, accessed May 6, 2024, https://lieu.house.gov/about/full-biography. ↩︎
24. Lieu, “Meet Ted.” ↩︎
25. Will Henshall, “How the House Bipartisan Task Force Is Thinking About AI,” TIME, February 21, 2024, accessed May 6, 2024, https://time.com/6727264/house-artificial-intelligence-task-force/. ↩︎
26. Lieu, “House Launches Bipartisan Task Force.” ↩︎
27. Lieu, “House Launches Bipartisan Task Force.” ↩︎
28. Greg Norman, “AI can predict political orientations from blank faces – and researchers fear ‘serious’ privacy challenges,” Fox News, April 23, 2024, accessed May 6, 2024, https://www.foxnews.com/politics/ai-can-predict-political-orientations-blank-faces-researchers-fear-serious-privacy-challenges ↩︎
29. “President’s News Conference,” Ronald Reagan Presidential Library, August 12, 1986, accessed May 6, 2024, https://www.reaganlibrary.gov/archives/speech/presidents-news-conference-23. ↩︎