Universities used EAB's Navigate software that incorporated students' race as a predictor variable in algorithms that labeled Black and other minority students as 'high risk' for dropping out at rates up to 4 times higher than White students.
More than 500 universities across the United States use education research company EAB's Navigate advising software to evaluate student success and dropout risk. Documents obtained by The Markup show that at least four out of seven schools incorporate race as a predictor variable, with two describing race as a 'high impact predictor.' Analysis of aggregate student risk data from four large public universities for fall 2020 revealed significant racial disparities: at University of Massachusetts Amherst, Black women were 2.8 times more likely to be labeled high risk than White women, and Black men 3.9 times more likely than White men. Similar patterns existed at University of Wisconsin-Milwaukee and Texas A&M University. Black students comprised less than 5% of UMass Amherst's undergraduate population but accounted for over 14% of students deemed high risk. The algorithms use historic student data from 2-10 years to train models that also consider SAT/ACT scores, high school percentile, and credits completed. The software includes a Major Explorer function that shows predicted risk scores across different majors, potentially steering students toward 'easier' programs. Students reported being unaware they were being algorithmically evaluated, and professors received little training on interpreting risk scores.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed