Amazon's automated workforce management systems at its JFK8 warehouse systematically tracked worker productivity and time off task, leading to inadvertent firings, stalled benefits, and communication failures that particularly affected minority workers during the COVID-19 pandemic.
Amazon deployed extensive AI-powered workforce management systems at its JFK8 fulfillment center on Staten Island, using algorithms to track worker productivity through metrics like 'rate' and 'time off task' (TOT). The systems monitored every minute of workers' shifts through handheld scanners, automatically identifying 'top offenders' for disciplinary action and generating termination recommendations. During the COVID-19 pandemic, these automated systems malfunctioned extensively, causing inadvertent firings, job abandonment notices sent to workers on approved leave, and disability payment stoppages. The systems particularly affected Black and Latino workers, who comprised over 60% of associates but faced firing rates 50% higher than white peers. Amazon's 'lights-out hiring' process used algorithms to hire workers with minimal human interaction, contributing to turnover rates of approximately 150% annually. The company hired over 770,000 hourly workers in 2019 while growing by only 150,000 employees total. Workers like Alberto Castillo, who suffered severe COVID-related brain damage, received automated return-to-work notices despite being medically incapacitated. The systems assumed worker fault for productivity issues and required managers to interrogate the highest TOT workers each shift, creating anxiety among employees who feared bathroom breaks would trigger disciplinary action.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed