AI capabilities have the potential to be exploited for personal gain at the expense of others via deception and manipulation. This can take various forms including cheating, fraud, scams, and the use of deepfakes for blackmail or humiliation. It is currently very difficult to distinguish human text from text that is AI-generated. This increases opportunities for cheating in settings where rewards depend on the communication of original thought. In academia, students may use AI to quickly generate essays or other coursework and claim it as their own. If students regularly and inappropriately rely on AI for their schooling, this could undermine academic integrity and genuine intellectual development. In science, researchers could use AI unscrupulously to produce professional outputs. If widely adopted, this practice could dilute the overall quality of scientific discourse.
Generative AI products may also be used to increase the reach and potency of various dishonest schemes. Advanced AI assistants can produce HTML, CSS, and other web development languages, allowing for the rapid creation of convincing fraudulent websites and applications at scale. In the context of social media, generative adversarial networks (GANs) have been used to create images of human faces that look authentic. AI models can also be trained on speech or writing data from a specific individual, allowing the model to impersonate someone very convincingly without consent. Scammers could use this capability to request sensitive information or financial aid by pretending to be a trusted contact. A particularly damaging type of abuse facilitated by deep fakes involves creating non-consensual sexual imagery with the intent to cause a subject social injury or manipulate them into performing desired actions. Even if a deep fake is exposed as inauthentic, it can continue to impact a person's life in significant ways through the loss of job opportunities, social isolation, and ongoing harassment or defamation.
Excerpt from the MIT AI Risk Repository full report
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
A viral AI-generated video falsely showed India's Finance Minister Nirmala Sitharaman endorsing a fraudulent high-return investment scheme, which was debunked by the government's fact-checking unit PIB Fact Check.
Developers: Unknown Deepfake Technology Developers, Unknown Voice Cloning Technology Developers
Deployers: Unknown Scammers
Malicious actors used AI deepfake technology to create a fraudulent video impersonating Albanian cardiologist Spiro Qirko and journalist Ilir Topi to illegally promote an unauthorized hypertension medication called 'Hyper Caps'.
Developers: Unknown Voice Cloning Technology Developers, Unknown Deepfake Technology Developers
Deployers: Unknown Scammers
A Florida man used AI to create a fabricated 3-second video showing people entering a police patrol car, then presented it to a deputy as evidence of a crime that never occurred, leading to his arrest on multiple charges including fabricating physical evidence.
Developers: Unknown Deepfake Technology Developers
Deployers: Alexis Martinez Arizala
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
193 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
192 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
183 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
182 shared governance docs
Requires the Secretary of Defense to develop a cybersecurity policy for AI/ML systems no later than 180 days after the act is passed. Develop a comprehensive review of the effectiveness of the AI/ML policies. Addresses potential security risks, implements methods to mitigate those risks, and establishes standard policy. Requires a comprehensive report of the threats and cybersecurity measures by August 31, 2026.
Amends California's Business and Professions Code to require large online platforms to detect and disclose provenance data of AI-generated content, prohibit GenAI hosting platforms from distributing non-compliant systems, and require capture device manufacturers to embed latent disclosures by default, with civil penalties of $5,000 per violation.
Requires large frontier developers to implement and publish frontier AI frameworks, assess catastrophic risks, and publish transparency reports; requires the Office of Emergency Services to establish reporting mechanisms for critical safety incidents and catastrophic risk assessments; establishes a consortium to develop a framework for the creation of CalCompute; creates civil penalties for violations of this chapter.