Algorithm

"Performance of model architecture, optimization algorithm, and training procedure: Different combinations of model architecture, optimization algorithm, and training procedure have different effects on its final performance (e.g., accuracy, generalization). These choices are independent of modeling choices (discussed in Section 4.6), where the ML practitioner translates a problem statement into an ML problem/task (e.g., by defining the input and output space). For example, a language model can be trained with either the causal or masked language modeling objective [52]. While the latter is suitable for text classification, it may be suboptimal for text generation. Additionally, some training procedures (e.g., domain adversarial training [74]) may improve the ML system’s ability to generalize to new domains with minimal extra training data but may hurt performance on the original domain. While accuracy on general benchmark datasets is often used to differentiate models, a better indicator of real-world efficacy is performance on similar applications, due to nuances in the target distribution and the tendency of state-of-the-art models to be optimized for leaderboards [61]."(p. 7)

"Beyond efficacy, it is also important to consider the reliability and resource intensiveness of the chosen ML algorithm, model architecture, and optimization technique combination in production scenarios. From an operational standpoint, a highly accurate system that is computationally intensive or failure-prone may be less desirable than a slightly less accurate one without those flaws."(p. 7)

"Explainability/transparency:Algorithmic opacity and unpredictability can pose risks and make it difficult to ensure accountability. While new mandated levels of transparency and explainability of algorithms are being demanded through the likes of the EU’s General Data Protection Regulation (GDPR) to tackle bias and discrimination, it can be at times impossible for the experts to interpret how certain outputs are derived from the inputs and design of the algorithm. This suggests the difficulty of assigning liability and accountability for harms resulting from the use of the ML system, as inputs and design rules that could yield unsafe or discriminatory outcomes cannot as easily be predicted. Therefore, a system that can explain its decision in the event of a mistake is often desirable in high-stakes applications. A mistake can take the form of an accident resulting from a decision, a denied loan, assigning different credit limits based on gender. While explainability on its own is insufficient to reduce biases in the system or make it safer, it may aid the detection of biases and spurious features, thereby reducing safety and discrimination risks when the flaws are rectified. Other use cases, such as judicial applications, may require such explainability due to their nature. However, not all machine learning algorithms are equal in this regard. Decision trees are often considered highly explainable since they learn human-readable rules to classify the training data, while deep neural networks are a well-known example of a black-box model. While there have been recent advances in explaining neural network predictions, researchers have also demonstrated the ability to fool attention-based interpretation techniques. This may allow developers to prevent the network’s predictions from being correctly interpreted during an audit. The choice of an ML algorithm and its training method, therefore, affects this aspect of algorithmic risk."(p. 7)