Robustness

"Scope of deployment environment: Similar to Section 4.1, the deployment environment’s scope determines the range of variation the ML system will be exposed to. For example, it may be acceptable for an autonomous robot operating in a human-free environment to be unable to recognize humans, but the same cannot be true for a similar robot operating in a busy town square. A larger range, therefore, usually necessitates either a more comprehensive dataset that can capture the full range of variation or a mechanism that makes the system robust to input variation. A broader scope may also increase the possibility of adversarial attacks, particularly when the system operates in a public environment."(p. 10)

"Mechanisms for handling of OOD inputs: Out-of-distribution (OOD) inputs refer to inputs that are from a distribution different from the training distribution. They include inputs that should be invalid, noisy inputs (e.g., due to background noise, scratched/blurred lenses, typographical mistakes, sensor error), natural variation (e.g., different accents, lens types, environments, grammatical variation), and adversarial inputs (i.e., inputs specially crafted to evade perception or induce system failure). Incorporating mechanisms that improve robustness (e.g., adversarial training) reduces robustness risk, but often comes with extra computational overhead during training or inference."(p. 10)

"Failure recovery mechanisms: In addition to functioning correctly in the presence of OOD inputs, system robustness also includes its ability to recover from temporary failure. An example of recovery is an autonomous quadrupedal robot regaining its footing without suffering physical damage after missing a step on the way down a staircase."(p. 10)