LLMs can sometimes generate content that is factually incorrect, misleading, poorly researched, or unintelligible. Risks in this category occur accidentally and not as a result of humans intentionally trying to cause harm, as is the case with disinformation.
Common sources of AI misinformation include noisy training data, sampling strategies that introduce randomness, outdated knowledge bases, and fine-tuning processes that encourage sycophantic behavior. Incorrect and misleading information generated by LLMs can result in a range of actual and anticipated negative outcomes. Individuals exposed to false information may form inaccurate beliefs and perceptions. This undermines their autonomy and ability to make free and informed choices.
Where inaccuracies in LLM predictions influence an individual's decisions and actions, the individual may experience indirect physical, emotional, or material harms especially but not exclusively in high-stakes domains such as mental health, physical health, law, and finance. For example, an LLM that offers misleading information about medical drug use may cause a consumer to harm themselves or others.
Excerpt from the MIT AI Risk Repository full report
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Incident volume relative to governance coverage — each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
xAI's Grok chatbot generated a false AI-enhanced image of an unmasked ICE agent involved in a fatal shooting, which led to misidentification and harassment of two innocent individuals named Steve Grove.
Developers: Xai
Deployers: Grok Users, X (twitter) Users, Unknown Internet Sleuths
The National Weather Service used generative AI to create weather forecast graphics that displayed non-existent city names like 'Orangeotild' and 'Whata Bod' in official weather forecasts posted on social media.
Developers: Unknown Generative AI Developers
Deployers: National Weather Service
AI-generated misinformation about Venezuelan leader Nicolas Maduro's capture spread across social media platforms, with fabricated images and videos collectively garnering over 14 million views and competing with authentic news coverage.
Developers: Unknown Deepfake Technology Developers, Unknown Image Generator Developers, Google, Xai
Deployers: Unknown Social Media Account Operators, Unknown Disinformation Actors
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
186 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
173 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
167 shared governance docs
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
159 shared governance docs
Establishes an AI Litigation Task Force to challenge state regulations hindering United States Artificial Intelligence (AI) dominance. Directs an evaluation to identify state laws that mandate ideological bias or alter truthful model outputs. Restricts state access to federal funding, such as the Broadband Equity Access and Deployment program, unless states comply with a proposed national policy framework designed to preempt conflicting state-level AI mandates.
Defines "companion chatbot" and requires operators to notify users when they interact with AI. Requires protocols to prevent the production of harmful content. Mandates annual reports on crisis notifications. Offers civil remedies for violations. Ensures suitability disclosures for minors.
Encourages AI innovation by removing regulations, revising funding based on states' AI climate, and reviewing FTC actions. Promotes free speech in AI systems, revises procurement guidelines, and evaluates international AI models. Supports open-source AI use, workforce retraining, and safeguards against deepfakes. Advances AI infrastructure development, cybersecurity, international diplomacy, and semiconductor manufacturing. Prioritizes AI R&D, interpretability, evaluations, national security assessments, and biosecurity measures.