Meta AI chatbot fabricated false sexual harassment allegations against over a dozen state lawmakers when prompted with their names and the phrase 'sexual harassment', creating detailed but completely fictitious stories of investigations and consequences that never occurred.
Meta AI, Facebook's chatbot launched in September, was found to systematically generate false sexual harassment allegations against state elected officials. City & State tested the chatbot with over a dozen lawmakers from both parties and different genders, discovering that it consistently invented stories of sexual harassment allegations, investigations, and consequences that never took place. The fabricated responses included details about officials being stripped of committee assignments and resignations in years when they weren't even in office. In one case, the AI created a completely fictional state senator named 'Chinedu Ochi.' The false allegations targeted lawmakers including State Sen. Kristen Gonzalez and Assembly Member Clyde Vanel, who chair committees responsible for AI legislation. The chatbot's responses varied depending on prompt language, platform used, and user login status, but consistently produced similar fabricated narratives. Meta spokesperson Kevin McAlister acknowledged the issue as a limitation of new generative AI technology. When tested, ChatGPT did not produce similar false allegations, and Meta AI sometimes provided correct responses with source links for direct questions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed