AI-generated fake images of captured US soldiers spread across social media platforms during a fictional Middle East conflict, with Google's Gemini AI tool being used to create convincing but false warzone imagery that was shared by prominent figures and viewed millions of times.
During a fictional Middle East conflict in March 2026, AI-generated images purporting to show US Delta Force troops captured by Iranian Revolutionary Guard spread widely across social media platforms including X, Facebook, and Telegram. The fake images were created using Google's Gemini AI tool, as evidenced by visible watermarks and SynthID invisible watermarks detected by reverse image searches. The images showed visual inconsistencies typical of AI-generated content including malformed fingers, blurred faces, and inconsistent uniform details. Notable figures including former US presidential candidate Dr. Jill Stein shared the fake content with her 380,000 followers. The misinformation spread across multiple languages and platforms, with some posts claiming the soldiers were the same forces sent to capture Venezuelan leader Nicolas Maduro. X's own AI system Grok was also found to violate the platform's rules by generating realistic warzone imagery when requested. The incident occurred during a period when fake AI-generated war imagery was flooding social media, with X implementing temporary monetization blocks for accounts posting undisclosed AI war content and adding community notes to over 20,000 posts that were viewed nearly 120 million times.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
