Monitoring protocols for potentially adversarial GPAI systems

Definitionp. 41

Similar to cybersecurity protocols, AI providers can prepare in advance for potential adversarial action from AI systems by developing and stress-testing protocols for potential adversarial action [179]. Such adversarial action may range from hiding code exploits, to strategic deception or loss of control related behaviors. AI systems may be adversarial due to malicious actors (e.g., data poisoning), or due to unintended issues of control (lack of alignment with human intent). Alongside improving the system alignment, external monitoring and restrictions may be deployed in advance to “catch” these adversarial model malfunctions before harm is done. Like with cybersecurity red teaming, any protocol which assumes adversarial action can be stress tested in advance. For example, this can be done by projecting the plausible capabilities and affordances of the adversarial actor (e.g., in the case of a tech company - a hacker, or a disgruntled employee). From this process, measures that better anticipate possible adversarial action can be deployed, and catch unsuccessful attempts at adversarial behavior. If such unsuccessful attempts are caught, they can be reported in incident reports, and serve as an input to further risk analysis. Depending on their severity, it may be advisable to recall the system if deployed.

Additional Informationp. 41-42

Examples of these protocols include [84]: • Flagging of suspicious frontier AI system code generation for human oversight, with the help of better proven (“trusted”) AI systems or automated keyword detection. • Using a better proven (“trusted”) AI system to edit the code produced by a frontier AI system if suspicious.• Monitoring of AI system with another instance of this same system, with filters to prevent collusion. • Restriction of access of the AI system to private information.

Part of