The University of Texas at Austin used an AI system called GRADE from 2013-2020 to evaluate PhD applications in computer science, which critics argue perpetuated bias against underrepresented groups by replicating historical admissions patterns.
The University of Texas at Austin's computer science department used a machine-learning system called GRADE (GRaduate ADmissions Evaluator) from 2013 to 2020 to help evaluate PhD applications. GRADE was created by UT faculty and graduate students to save time reviewing the growing number of applications, which had increased from 250 in 2000 to over 1,200 by recent years. The system was trained on historical admissions data and assigned scores from 0-5 to predict whether applicants would be accepted. It analyzed factors including GPA, university prestige (categorized as 'elite,' 'good,' or 'other'), and letter of recommendation language. The system reduced full reviews by 71% and total review time by 74%. Every application was still reviewed by at least one human, but sometimes only one instead of multiple reviewers as was done previously. The system came under criticism in November 2020 when graduate students, particularly Yasmeen Musthafa, raised concerns on Twitter about potential bias against underrepresented groups. Critics argued that training on historical data would perpetuate existing biases, and that the system undervalued personal statements and recommendation letters that could help marginalized applicants. UT Austin announced it had discontinued GRADE in early 2020, citing both bias concerns and maintenance difficulties.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed