System-theoretic process analysis (STPA)

Definitionp. 36

STPA is a method to assess the utility of implemented controls against a particular risk within a complex system. Unlike bow-tie analysis, STPA factors in the interactions between components as events that can cause the risk in question [107].

Additional Informationp. 37

First, the system and its boundaries to the environment are defined. The system is primarily delineated from the environment because there is at least some partial control over it. Second, several items are enumerated, including (i) unwanted risk events (“losses”), (ii) system states that cause losses (“system-level hazards”), and (iii) system states that do not cause losses (“system-level constraints”). Third, a diagram mapping the system, environment, their different controls, and the interactions between these elements is created. This diagram must be comprehensive in listing the different losses and possible interactions that can cause each loss. Finally, the diagram can be used to identify “unsafe control actions” (UCAs), which are the causal pathways between a control and system-level hazards, including all interactions involved. For example, in the context of text-to-image models, losses may include ‘loss of diversity’ and ‘loss of quality’; hazards may include ‘low quality text-image pairs within training dataset’ and ‘harmful content within training dataset’; and the controls may include human controllers and automated controllers (e.g., annotators, data owners, data crawlers) [165]. Subsequent analysis may result in identification of UCAs such as ‘current data filtering actions’ and neglecting current filter thresholds’ which are linked to specific hazards. Specific actions that counter or prevent such UCAs can reduce the associated losses.

Part of