Stanford Medicine used an algorithm to distribute COVID-19 vaccines that prioritized senior faculty over frontline residents, leading to only 7 out of 1,300 residents being selected for the first wave despite their high exposure risk.
In December 2020, Stanford Medicine deployed an algorithm to allocate its first 5,000 doses of the Pfizer COVID-19 vaccine among staff members. The algorithm considered employee-based variables including age, job-based variables related to COVID-19 prevalence by role and department, and California Department of Public Health guidelines. However, the system resulted in only 7 out of approximately 1,300 medical residents and fellows being selected for the first wave of vaccinations. Instead, the algorithm prioritized senior faculty members, including some who worked remotely from home, pathologists, radiologists, and other staff with lower patient exposure risk. Medical residents, who are frontline workers regularly treating COVID-19 patients and had been asked to volunteer for additional ICU coverage, were disadvantaged because they lacked assigned locations in the algorithm and were typically younger. On December 18, 2020, about 100 healthcare workers staged a protest during the planned vaccination rollout, demanding transparency and priority for frontline workers. Stanford Medicine leadership apologized, acknowledging the algorithm had failed and promising immediate revision of the distribution plan. The incident highlighted challenges in algorithmic decision-making for resource allocation during the pandemic.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed