A speech-to-text AI system used by the Genoa prosecutor's office incorrectly transcribed 'finanziamenti leciti' (legal financing) as 'finanziamenti illeciti' (illegal financing) during a corruption investigation, potentially influencing the entire inquiry until the error was discovered during re-examination.
On May 26, 2024, in the office of preliminary investigation judge Paola Faggioni in Genoa, an audio recording of Roberto Spinelli's interrogation was re-examined after his lawyers requested clarification of a crucial detail in a corruption investigation involving Liguria region president Giovanni Toti. The transcript showed Spinelli had spoken of 'illegal financing' given to Toti, which would have confirmed prosecution allegations and was highlighted in media coverage. However, upon re-listening to the audio, it was discovered that Spinelli had actually said the exact opposite: 'legal financing.' The error originated from a speech-to-text software system used by the prosecutor's office to automatically transcribe interrogations. These AI transcription systems have been adopted by many Italian courts and prosecutor's offices in recent years due to budget cuts that reduced staff and transcribers. The systems are trained on thousands of hours of audio from TV programs, court hearings, and political debates, creating probabilistic models that can be influenced by cultural biases in the training data. Legal experts note that if 'financing' is frequently associated with illegal conduct in the training data, the system may predict 'illegal' as more probable than 'legal' when transcribing unclear audio.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed