Facebook's recommendation algorithm began promoting AI-generated spam posts depicting fake religious imagery, disabled individuals, and impossible objects, causing user confusion and potential financial fraud as people believed the content was real.
Facebook users reported a surge in AI-generated spam posts being recommended by the platform's algorithm in recent months. The content included fake religious imagery with captions like 'Close your eyes 70% and see magic,' emotionally exploitative images of disabled individuals asking for birthday wishes, and impossible architectural or handicraft items. Researchers from Georgetown and Stanford universities analyzed over 100 Facebook pages posting AI content dozens of times daily and found many were engaging in scams and spam. Some posts accumulated hundreds of thousands or millions of interactions from users who believed they were real. The pages included classic spam linking to ad revenue websites and scammers advertising non-existent AI-generated products. Meta CEO Mark Zuckerberg revealed that recommended posts now account for about 30% of users' feeds, double the previous amount. Users reported being frustrated with the platform and some considered leaving entirely. One user mentioned having to discourage her mother from attempting to purchase fake woodwork items seen on Facebook.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed