Google's Smart Reply feature for Gmail repeatedly suggested inappropriate responses like 'I love you' and 'Sent from my iPhone' during development and testing phases before being corrected by engineers.
Google developed Smart Reply, an AI-powered feature for Gmail that analyzes email content and suggests three brief response options to help users quickly reply to messages. The system uses machine learning and neural networks to analyze billions of Gmail conversations and generate contextually appropriate responses. During development and early testing phases, the AI system exhibited problematic behavior by frequently suggesting inappropriate responses. Early prototypes had a 'propensity to respond with I love you to seemingly anything' according to Google's director of product management Ajit Varma. The system also suggested 'Sent from my iPhone' as a response option, having identified this as a common phrase in emails without understanding it was an automatic signature. Google engineers discovered these issues during internal testing and modified the algorithms before public release. The feature launched in Google's Inbox app in 2015 and later became available to Gmail's 1.4 billion users. Around 10 percent of Gmail responses are now generated using Smart Reply. Some users reported feeling that automated responses made communication feel less personal and human.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Pre-deployment
Occurring before the AI is deployed