Racial bias built into computer software used for lung function testing is leading to fewer Black patients getting diagnosed and treated for breathing problems, with the study suggesting 40% more Black male patients might have been diagnosed if race-based adjustments were removed.
A study published in JAMA Network Open examined racial bias in medical diagnostic software used for lung function testing. The research analyzed more than 2,700 Black men and 5,700 white men tested by University of Pennsylvania Health System doctors between 2010 and 2020. The AI-powered diagnostic software uses race-based adjustments that raise the threshold for diagnosing breathing problems in Black patients, making them less likely to receive medications, medical procedures, or lung transplants. The study found that removing these race-based calculations could result in nearly 400 additional cases of lung obstruction or impairment being diagnosed in Black men - representing about 40% more diagnoses. The software analyzes spirometry tests that measure how much and how quickly a person can inhale and exhale, then generates reports that help doctors diagnose conditions like asthma, chronic obstructive pulmonary disorder, or lung scarring. These race-based assumptions in medical algorithms stem from centuries-old beliefs that Black people's lungs were innately different from white people's lungs. The American Thoracic Society has recommended replacing race-focused adjustments, though implementation across different hospitals using various software versions may take time.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed