Microsoft replaced approximately 77 human journalists and editors with AI systems for news curation across MSN and Microsoft News platforms, which subsequently led to the AI confusing two mixed-race members of the band Little Mix in a story about racism.
In late May 2020, Microsoft announced layoffs of dozens of journalists, editors, and other workers at MSN and its news divisions as part of a push to automate journalism using AI. Approximately 50 jobs were affected in the US and 27 in the UK, with around 77 total editorial workers being replaced by AI algorithms that scan the internet for news articles to highlight on Microsoft News, MSN, and Microsoft Edge browser homepages. The laid-off workers had been responsible for curating news stories, selecting articles from other outlets, editing content and headlines, and ensuring adherence to strict editorial guidelines to avoid violent or inappropriate content. Within a week of the AI deployment, the automated system made a significant error by illustrating a story about Little Mix singer Jade Thirlwall's experience with racism using a photo of her bandmate Leigh-Anne Pinnock, both of whom are mixed-race. Thirlwall criticized the mistake on Instagram, noting that such confusion between the two women of color happens frequently and is offensive. Microsoft stated the error was caused by an experimental feature that selected alternate images rather than algorithmic bias, but acknowledged it was exactly the type of mistake human editors are supposed to catch. The remaining human staff were reportedly warned that critical coverage of the automated systems might be selected by the AI for the homepage and instructed to remove such stories.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed