Researchers demonstrated that GPT-2 and GPT-3 language models memorize and can be prompted to output verbatim training data including personally identifiable information, copyrighted content, and other sensitive material from their training datasets.
Computer scientists from Stanford University, UC Berkeley, OpenAI, and Google demonstrated that large language models GPT-2 and GPT-3 memorize and regurgitate training data when prompted. Using a two-step extraction attack, researchers generated 600,000 samples from GPT-2 and found 604 samples containing verbatim text from the training set. The extracted data included personally identifiable information (names, phone numbers, email addresses), IRC conversations, source code, and copyrighted content. About 13% of memorized examples contained personal contact information. GPT-3 was shown to reproduce about 240 words from Harry Potter verbatim, and GPT-2 could output 264 lines of Bitcoin client code and an entire 1446-line game configuration file. The researchers found that larger models are more vulnerable to such extraction attacks. The memorized personal information appeared in inappropriate contexts, such as fictitious IRC conversations about transgender rights using real usernames from leaked GamerGate logs. OpenAI subsequently began developing content filtering systems to prevent GPT-3 from outputting personal information through its commercial API.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed