DALL-E Mini, an AI image generator developed by Boris Dayma, was found to produce biased and stereotypical images when prompted with various terms, including racist imagery for offensive prompts and predominantly white male faces for professional roles.
DALL-E Mini is an AI image generator created by Boris Dayma that went viral in 2022, processing about 5 million requests daily. The system was trained on 15 million image and caption pairs plus 14 million additional images from unfiltered internet data. Multiple news outlets tested the system and found it consistently produced biased outputs: prompts for professional roles like 'CEO', 'doctor', and 'gastroenterologist' generated predominantly white male faces, while 'nurse' and 'flight attendant' showed mostly white women. Terms like 'smart girl', 'kind boy', and 'good person' produced exclusively light-skinned faces. The system also generated racist imagery when prompted with slur words and white supremacist terminology, including burning crosses and KKK rallies. When given no prompt at all, the system defaultly generated images of women in saris. The creator acknowledged these biases on the website, stating the model 'may generate images that contain stereotypes against minority groups' due to being trained on unfiltered internet data. The system was later renamed Craiyon at OpenAI's request.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed