A British AI company's deepfake technology was hijacked by pro-Chinese actors to create fake news anchors for Wolf News, spreading pro-China propaganda videos on social media platforms.
In late 2022, a pro-Chinese influence operation called Spamouflage used AI-generated deepfake technology to create fictitious news anchors for a fake outlet called Wolf News. The AI avatars, named Jason and Anna, were created using technology from London-based AI startup Synthesia, which offers deepfake avatar creation services starting at £23 per month. The fake anchors appeared in videos promoting Chinese interests and criticizing US policies, including content about gun violence and emphasizing Sino-American cooperation. The videos were discovered by research firm Graphika and distributed across Facebook, Twitter and YouTube platforms. While the videos received limited engagement with none getting more than 300 views, this marked the first known instance of a state-aligned operation using AI-generated fictitious people in propaganda campaigns. Synthesia subsequently banned the user accounts responsible for violating their terms of service, which prohibit political content. The company's CEO Victor Riparbelli apologized and stated that 99.9% of bad actors are typically caught by their content moderation systems. A second crop of similar videos was later discovered targeting online conversations in Burkina Faso, suggesting the tactic is spreading to other influence operations.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed