Google's AI Overview feature generated numerous inaccurate and potentially harmful responses to user queries, including recommending adding glue to pizza sauce, eating rocks for nutrients, and spreading false information about political figures.
Google deployed AI Overview, a new search feature that generates AI-powered summaries of search results, broadly to users in the United States in May 2024. The feature uses Google's Gemini large language model to scan web content and provide AI-generated responses above traditional search results. Shortly after launch, users discovered numerous problematic outputs including dangerous advice like mixing glue into pizza sauce, eating rocks daily for vitamins, and using gasoline in cooking. The system also spread misinformation, falsely claiming Barack Obama was the first Muslim president and providing incorrect historical information about various presidents. Many erroneous responses appeared to originate from satirical sources, joke Reddit posts, and unreliable content that the AI could not distinguish from factual information. Google acknowledged the issues and implemented over a dozen technical updates, reducing the frequency of AI Overviews for certain query types. The company stated that problematic examples came from 'generally very uncommon queries' but faced criticism for deploying the technology at scale before adequately addressing its reliability issues. The incident affected millions of users who had access to Google search, which controls over 80 percent of the global search market.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed