This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Galileu AI system, developed by the Brazilian Regional Labor Court, successfully detected and prevented a prompt injection attack in a legal petition that contained hidden instructions attempting to manipulate the AI's analysis.
The Galileu AI tool, developed by the Regional Labor Court of the 4th Region (TRT-RS) and nationalized by the Superior Council of Labor Justice (CSJT), detected a prompt injection attack in an initial petition processed by the 3rd Labor Court of Parauapebas (PA) on May 12th. The AI system identified hidden text segments containing instructions directed at the artificial intelligence itself, which aimed to make the system contest the petition superficially and not challenge documents, regardless of the command it received. This technique is known as prompt injection. Upon detection, the Galileu system issued a highlighted alert to the user with technical identification of the occurrence and prevented the maliciously inserted content from being processed by the tool. The system limited itself to reporting the technical fact without qualifying the conduct or proposing procedural measures. Judge Luiz Carlos de Araújo Santos Junior examined the content identified by the system before taking any action, maintaining human oversight as required by Brazilian judiciary regulations. The incident demonstrates the importance of using institutional AI tools with proper security measures, as controlling such attacks requires specialized techniques for identification according to technology officials.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
