WeChat's AI translation system incorrectly translated the neutral Chinese phrase 'hei laowai' (black foreigner) into the N-word when used in negative contexts, exposing users to racist language.
WeChat, a Chinese messaging app with nearly 1 billion users, experienced an AI translation error where its neural machine translation system incorrectly translated the neutral Chinese phrase 'hei laowai' (black foreigner) into the N-word when used in negative contexts. The issue was discovered by Ann James, a black American theater director in Shanghai, when she used WeChat's translation feature to read Chinese responses and saw 'The n----- is late' instead of the neutral phrase. Testing by local media outlets found that the app translated the phrase neutrally in positive contexts but used the racial slur when combined with negative terms like 'late' or 'lazy.' WeChat, owned by Tencent (valued at $275 billion), apologized for the inappropriate translation and immediately fixed the problem after receiving user feedback. The company explained that the app used neural machine translation that learns from large datasets, which can inadvertently pick up offensive associations and language patterns without human oversight.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed