Skip to main content
MIT AI Risk Initiative
BackJailbreak in LLM Malicious Use - Poisoning Training Data
Home/Risks/Wang et al. (2025)/Jailbreak in LLM Malicious Use - Poisoning Training Data
Toxicity in LLM Malicious Use
Jailbreak in LLM Malicious Use - Backdoo…

Jailbreak in LLM Malicious Use - Poisoning Training Data

A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy

Wang et al. (2025)

SourceDOI
Sub-category
Risk Domain
2Privacy & Security
2.2AI system security vulnerabilities and attacks

Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

"In the data collecting and pre-training phase, malicious adversaries can Jailbreak LLMs through poisoning their training data to make the model to output harmful content."(p. 21)

Entity— Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent— Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal

Other

Without clearly specifying the intentionality

Timing— Whether the risk is pre- or post-deployment

Pre-deployment

Occurring before the AI is deployed

Post-deployment

Occurring after the AI model has been trained and deployed

Other

Without a clearly specified time of occurrence

Other risks from Wang et al. (2025) (11)

Privacy - Membership Inference Attack (MIA)

2.2 AI system security vulnerabilities and attacks
HumanIntentionalPost-deployment

Privacy - Data Extraction Attack (DEA)

2.2 AI system security vulnerabilities and attacks
HumanIntentionalPost-deployment

Privacy - Prompt Inversion Attack (PIA)

2.2 AI system security vulnerabilities and attacks
HumanIntentionalPost-deployment

Privacy - Attribute Inference Attack (AIA)

2.2 AI system security vulnerabilities and attacks
HumanIntentionalPost-deployment

Privacy - Model Extraction Attack (MEA)

2.2 AI system security vulnerabilities and attacks
HumanIntentionalPost-deployment

Hallucination

3.1 False or misleading information
AI systemUnintentionalPost-deployment
View all 11 risks from this paper →