Data operations

Data sourcing

Data is crucial to both AI systems and algorithm development and operations. The AI System Owner Sources should ensure that the organization defines and documents AI system data sources. The AI System Owner should ensure that the organization has the right to process the data. The Algorithm Owner should ensure that the organization defines and documents training, validation, and operational data sources. The Algorithm Owner should ensure that the organization has the right to process the algorithm training, validation, and operational data.

Data ontologies, inferences, and proxies

Data resources contain various categories of data. The categories reflect explicit or implicit data ontologies. Data ontologies consist of entity taxonomies (what entities are assumed to exist) and models of entity relationalities and causality (how the entities relate to each other). Data ontologies may have significant implications on how algorithms and AI systems function, what risks they create and to whom, and what entities and how the AI system affects them. In advanced machine learning approaches, data ontologies are complex as the source data ontologies combine with the non-representational sensemaking inherent to the approaches. Understanding the ontologies may only be possible by analyzing algorithm outputs. The AI System Owner should ensure that the organization 10) adequately understands the AI system data ontology, 11) has explored the risks related to possible inconclusive evidence, system, and discrimination risk, and 12) develops and implements measures to minimize and mitigate possible data-related risks. The Algorithm Owner should ensure that the organization 1) adequately understands the algorithm data ontology, 2) adequately understands what inferences are drawn on the data and what proxies are created when the organization uses a machine learning approach to develop an algorithm, 3) has explored the risks related to possible inconclusive evidence, system bias, and discrimination risks the data ontology may create, and 4) develops and implements measures to minimize and mitigate possible data ontologyrelated risks. In particular, if the AI system makes decisions that affect natural persons, the AI system owner should ensure that the organization conducts a comprehensive assessment of the AI system’s discrimination, misidentification, and cultural sensitivity risks. The AI System Owner and Algorithm owner should ensure that the residual risks are acceptable and align with the organization's values and risk tolerance.

Data quality metrics

The organization can only ensure desired AI system performance by designing appropriate metrics to evaluate data quality. The AI System Owner should ensure that the organization defines and documents data quality metrics for assessing the quality of the data the AI system uses. The Algorithm Owner should ensure that the performance metrics align with the organization's values and risk tolerance.

Data quality assurance

Adequate data quality is a crucial precondition to all AI system operations. The AI System Owner should ensure that the organization designs and entrenches appropriate workflows and technical arrangements for 1) gathering and producing information on data quality, and 2) ensuring that the data (including the training, validation, and testing data) is of adequate quality and sufficiently relevant, complete, and representative. Data quality analyses should also include an analysis of additional data needs.

Data preprocessing

Training, testing, and operation undergo preprocessing in many AI systems. The AI System Owner and Algorithm Owner should ensure that the organization designs and implements appropriate workflows and technical interfaces for effective and appropriate data preprocessing. As training data, validation data, and operational data often differ qualitatively, the Algorithm Owner should ensure that the organization understands the differences and designs and implements appropriate workflows and interfaces for preprocessing each data category. The AI System Owner should ensure that the data preprocessing process aligns with the organization's values and risk tolerance

Data quality monitoring

The AI System Owner should ensure that the organization implements the planned data quality processes. If the data quality control processes disclose a breach of data quality standards, data drift, or indicate a value or risk tolerance misalignment, the AI System Owner should initiate appropriate measures to address the breach or regain alignment.

Data health check design

Data resources may be subject to deterioration over the medium and long term. In addition, the business, operational, IT, and regulatory environments and stakeholder pressures will change over time. These processes may jeopardize data access or data quality and lead to unacceptable risks. The AI System Owner should ensure that the organization designs processes for regular comprehensive reviews of the AI system resources (Data health checks) to ensure that the data-related risks are acceptable and align with the organization's values and risk tolerance. The AI System Owner should ensure that the organization defines, documents, and entrenches workflows and technical interfaces to review 1) AI system and algorithm data sources, 2) data preprocessing practices, 3) data quality, and 4) data ontology, inferences, and proxies.

Data quality monitoring design

Monitoring AI system data quality is crucial to ensuring that the AI system sustains the desired level of operational performance. Data quality monitoring must be systematic and metricsbased to achieve consistency over time. The AI System Owner should ensure that the organization defines, documents, and entrenches workflows and technical interfaces to facilitate the monitoring of data quality. In particular, the AI System Owner should identify anomalous data entries and data drift. including 1) automated or manual production and reporting of data quality indicators, alarm thresholds, and 2) workflows that allocate monitoring responsibilities. 3) workflows to address issues detected during regular monitoring. The Algorithm Owner should ensure that the data quality design process aligns with the organization's values and risk tolerance.

Data health checks

The Algorithm Owner should ensure that the organization performs the regular planned health checks. The reviews should assess whether the AI system data resources and data-related processes align with the organization's values and risk tolerance. If a review discloses a misalignment, the AI System Owner should initiate appropriate measures to regain alignment.