California's first online bar exam flagged 3,190 out of 9,301 test takers (34%) for potential cheating using ExamSoft's AI-powered proctoring software, leading to widespread allegations against test takers for vague infractions like looking away from cameras or having prohibited items.
In October 2020, California administered its first online bar exam using ExamSoft's AI-powered remote proctoring software due to the COVID-19 pandemic. Of the 9,301 people who took the exam, the AI system flagged 3,190 test takers (approximately 34%) for potential cheating violations. The AI monitored test takers through webcams and flagged various behaviors including having food, drinks, or electronic devices, gazing off-screen, or technical issues like no audible sound detection. The California Committee of Bar Examiners sent 'Chapter 6 Notices' to flagged individuals with vague allegations such as 'facial view of your eyes was not within view of the camera for a prolonged period of time' or 'no audible sound was detected' despite test takers being instructed not to make noise. Lawyers representing affected test takers report that many clients received accusations they could not understand or identify with their actual behavior. The flagged individuals face potential requirements to retake the exam or prove they were not cheating, with some reports suggesting they must respond to allegations without access to the video evidence. The review process was initially planned to be completed by December 18 but has been extended due to the volume of cases.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed