OpenAI discovered Chinese security operations using AI tools to build surveillance systems for monitoring anti-Chinese content on Western social media and conducting disinformation campaigns targeting Chinese dissidents and promoting anti-American content.
OpenAI reported on Friday that it had uncovered evidence of Chinese security operations using artificial intelligence to build surveillance tools and conduct influence campaigns. The primary operation, called 'Peer Review,' involved Chinese actors using ChatGPT to debug code for an AI-powered surveillance tool designed to monitor anti-Chinese posts on Western social media platforms including X, Facebook, YouTube, Instagram, Telegram, and Reddit. The tool, named 'Qianyue Overseas Public Opinion AI Assistant,' was built using Meta's open-source Llama model and was designed to collect real-time reports about anti-Chinese protests and share insights with Chinese authorities. A separate campaign called 'Sponsored Discontent' used OpenAI's technologies to generate English-language posts criticizing Chinese dissident Cai Xia and Spanish-language articles critical of the United States that were published in Latin American news outlets. OpenAI also identified a Cambodia-based campaign using AI to generate social media comments for 'pig butchering' romance scams. The company banned the associated accounts and disrupted several other malicious campaigns from North Korea, Iran, and other actors. Ben Nimmo, OpenAI's principal investigator, stated this was the first time they had uncovered an AI-powered surveillance tool of this kind.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed