This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
AI capabilities have the potential to be exploited for personal gain at the expense of others via deception and manipulation. This can take various forms including cheating, fraud, scams, and the use of deepfakes for blackmail or humiliation. It is currently very difficult to distinguish human text from text that is AI-generated. This increases opportunities for cheating in settings where rewards depend on the communication of original thought. In academia, students may use AI to quickly generate essays or other coursework and claim it as their own. If students regularly and inappropriately rely on AI for their schooling, this could undermine academic integrity and genuine intellectual development. In science, researchers could use AI unscrupulously to produce professional outputs. If widely adopted, this practice could dilute the overall quality of scientific discourse.
Generative AI products may also be used to increase the reach and potency of various dishonest schemes. Advanced AI assistants can produce HTML, CSS, and other web development languages, allowing for the rapid creation of convincing fraudulent websites and applications at scale. In the context of social media, generative adversarial networks (GANs) have been used to create images of human faces that look authentic. AI models can also be trained on speech or writing data from a specific individual, allowing the model to impersonate someone very convincingly without consent. Scammers could use this capability to request sensitive information or financial aid by pretending to be a trusted contact. A particularly damaging type of abuse facilitated by deep fakes involves creating non-consensual sexual imagery with the intent to cause a subject social injury or manipulate them into performing desired actions. Even if a deep fake is exposed as inauthentic, it can continue to impact a person's life in significant ways through the loss of job opportunities, social isolation, and ongoing harassment or defamation.
Excerpt from the MIT AI Risk Repository full report
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Scammers used AI to create deepfake videos of Italian TV journalist Safiria Leccese offering fraudulent personal loans, leading to financial losses for multiple victims who believed the fake content.
Developers: Deepfake Technology Developers, Voice Cloning Technology Developers
Deployers: Scammers
Scammers used AI voice cloning technology to impersonate a woman's daughter, claiming she was kidnapped by a Mexican drug cartel, and successfully defrauded the mother of $5,400 through wire transfers.
Developers: Deepfake Technology Developers, Voice Cloning Technology Developers
Deployers: Scammers, Fake Kidnapping Scammers
Two men, Cornelius Shannon and Arturo Hernandez, were arrested for using AI to create and distribute non-consensual deepfake pornography of celebrities and private individuals, with their content receiving millions of views online.
Developers: Deepfake Technology Developers
Deployers: Arturo Hernandez, Deepfake Creators
11.9% of 210 experts selected this risk in their top 3 concerns
Harm Severity (2025-2030)
Business as Usual
Pragmatic Mitigations
11% extreme risk under current trajectory → 9% with mitigations
Actor Vulnerability & Responsibility
| Actor | Vuln | Resp |
|---|---|---|
| Developer (GP) | 3 | 5 |
| Deployer | 4 | 4 |
| Governance | 3 | 4 |
| AI User | 5 | 3 |
| Developer (Spec) | 4 | 4 |
| Infrastructure Provider | 3 | 4 |
| Stakeholder | 5 | 2 |
Sector Vulnerability
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
195 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
193 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
184 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
184 shared governance docs
Requires the Secretary of Defense to develop a cybersecurity policy for AI/ML systems no later than 180 days after the act is passed. Develop a comprehensive review of the effectiveness of the AI/ML policies. Addresses potential security risks, implements methods to mitigate those risks, and establishes standard policy. Requires a comprehensive report of the threats and cybersecurity measures by August 31, 2026.
Amends California's Business and Professions Code to require large online platforms to detect and disclose provenance data of AI-generated content, prohibit GenAI hosting platforms from distributing non-compliant systems, and require capture device manufacturers to embed latent disclosures by default, with civil penalties of $5,000 per violation.
Regulates AI health advice by prohibiting unlicensed use of terms implying licensed health professionals' roles. Requires health care boards to enforce compliance. Prohibits implying licensed care through AI advertising or functionality. Penalizes each misuse of terms as separate violations.
