The PULSE image upscaling algorithm, which uses StyleGAN to generate high-resolution faces from pixelated inputs, consistently produced white faces when given low-resolution images of people of color, including Barack Obama, Lucy Liu, and Alexandria Ocasio-Cortez.
PULSE is an AI algorithm developed by researchers at Duke University that uses StyleGAN (created by NVIDIA) to upscale pixelated portrait images into high-resolution faces. The system works by generating completely new faces that, when pixelated, match the original low-resolution input rather than enhancing the original image. When the tool became widely accessible online in June 2020, users discovered significant racial bias in its outputs. Testing with pixelated images of prominent people of color, including Barack Obama, actress Lucy Liu, and congresswoman Alexandria Ocasio-Cortez, consistently produced white faces in the high-resolution outputs. The researchers acknowledged that PULSE produces white faces much more frequently than faces of people of color, attributing this bias to the StyleGAN training dataset which contains 70,000 images scraped from Flickr. Analysis showed StyleGAN generates white people 72.6% of the time compared to just 10.1% for Black people and 3.4% for Indian people. The bias went unnoticed during development because the team tested on the CelebA dataset, which is 90% white people. The incident sparked broader discussions about AI bias in the research community, with experts like Deborah Raji and Yann LeCun debating whether the problem stems purely from biased datasets or requires addressing wider social injustices in AI development.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed