Amazon's 'Amazon's Choice' recommendation algorithm promoted products with manipulated reviews, including recycled reviews from different products and fake positive reviews, misleading customers about product quality.
Amazon deployed an algorithmic recommendation system called 'Amazon's Choice' that automatically selects and promotes products to customers based on factors including star ratings and reviews. The system was designed to help customers make quick purchasing decisions in Amazon's massive marketplace and was integrated into both the Amazon website and Alexa voice assistant. However, investigations revealed that the algorithm was promoting products whose sellers had engaged in review manipulation, including recycling reviews from different products and purchasing fake positive reviews. The system failed to detect these deceptive practices, leading to customers being nudged toward potentially inferior products. One specific case involved customer R.D. who left a one-star review for a faulty pet hair vacuum, but after receiving a refund, her negative review was removed without consent and her attempt to republish it was never approved, while the product retained its 'Amazon's Choice' badge. The Federal Trade Commission has stated that review recycling constitutes deceptive advertising under the FTC Act. Amazon has acknowledged investing heavily in detecting inauthentic reviews but the manipulation persists, with the algorithm unable to detect nuances in review manipulation or accurately assess customer sentiment.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed