Development versions of iRobot Roomba J7 robot vacuums captured intimate images of users in their homes, which were then sent to Scale AI for data annotation and subsequently leaked by Venezuelan gig workers to social media groups.
In fall 2020, development versions of iRobot's Roomba J7 series robot vacuums captured private images from homes around the world, including intimate scenes of people in various states of undress and children. These images were sent to Scale AI, a data annotation company that contracts workers globally to label training data for AI systems. The images included 15 screenshots that MIT Technology Review obtained, showing scenes like a young woman on a toilet and a young boy on a hallway floor, with faces clearly visible. iRobot confirmed these came from special development robots with hardware and software modifications not present in consumer products, given to paid collectors who signed agreements acknowledging video recording. The devices had green stickers reading 'video recording in progress.' Scale AI contracted Venezuelan gig workers through its Remotasks platform to label the images as part of 'Project IO.' These workers then posted the images to closed social media groups on Facebook and Discord, violating non-disclosure agreements. iRobot has shared over 2 million images with Scale AI and other annotation platforms to train computer vision algorithms for object recognition and navigation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed