Microsoft's Windows Recall AI app, which takes screenshots of user activity for later search, was found to still capture sensitive information like credit card numbers, passwords, and Social Security numbers despite having filters designed to prevent this.
Microsoft's Windows Recall is an AI-powered app exclusive to Copilot+ PCs that automatically takes screenshots of user activity to enable later searching. The app was announced in summer 2024 but has faced repeated delays due to security concerns. Despite having a 'Filter sensitive information' feature enabled by default that is supposed to prevent capture of sensitive data like credit card numbers and passwords, testing by The Register found the filter fails in many cases. The testing was conducted on a Lenovo Yoga Slim 7x Copilot+ PC and revealed that while the filter successfully excluded some financial data and passwords, it captured bank home pages showing balances and deposits, failed to filter credit card information when certain contextual words were removed, and captured Social Security numbers when abbreviated differently. The app also captured screenshots of documents containing passwords when they weren't explicitly labeled as such. Microsoft has acknowledged this is a preview app and stated they are working to improve the filter functionality, but the app is being promoted during Windows setup on new PCs. The security concerns are compounded by the fact that Recall screenshots can be accessed by anyone with the user's PIN code, even remotely.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed