Goldman Sachs' Apple Card algorithm gave significantly higher credit limits to male applicants compared to equally qualified female applicants, prompting regulatory investigation after viral social media complaints.
In November 2019, software developer David Heinemeier Hansson publicly complained on Twitter that Goldman Sachs' Apple Card algorithm gave him a credit limit 20 times higher than his wife, despite her having a higher credit score and their shared financial circumstances including joint tax returns and community property status. Apple co-founder Steve Wozniak reported a similar experience with his wife receiving a credit limit 10 times lower than his. The complaints went viral and prompted filmmaker Lexi Alexander to conduct an informal test with 12 friends (6 men, 6 women), which confirmed the pattern of men receiving better offers despite women having perfect credit and high incomes. The New York Department of Financial Services launched an investigation into Goldman Sachs' credit card practices following these allegations. Goldman Sachs denied making decisions based on gender factors but acknowledged they evaluate applications individually rather than considering joint finances. The incident highlighted broader issues with algorithmic bias in financial services, where machine learning systems can perpetuate historical discrimination patterns even without explicitly using protected characteristics as inputs. In March 2021, the investigation concluded that Goldman had not violated fair lending laws, though regulators noted the need for modernizing credit scoring and anti-discrimination regulations.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed