+AI & Autonomous Technologies Development Practices
---+AI & Autonomous Technologies Transparency
---+AI & Autonomous Technologies Implementation Documentation
---+AI & Autonomous Technologies Human Domain Knowledge Reliance
|
AI & Autonomous Technologies Development Practices
Description
Measures exist to ensure Artificial Intelligence (AI) and Autonomous Technologies (AAT) are designed and developed to:
(1) Achieve an appropriate level of accuracy, robustness and cybersecurity;
(2) Perform consistently in those respects throughout the AAT system's lifecycle; and
(3) Be effectively overseen by competent individuals.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Document accuracy and robustness requirements before adopting AI
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ AI design requirements checklist covering accuracy, robustness, cybersecurity
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI development standards
∙ Security-by-design requirements for AI
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI development security standards
∙ Formal SDLC integration
∙ Security testing requirements
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI development framework
∙ AI security standards (NIST AI RMF, ISO 42001)
∙ DevSecOps integration for AI
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include:
▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation.
▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control, risk assessments, etc.).
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
▪ AAT is regarded as a technology and governed by the entity's existing IT governance practices.
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide oversight of AAT-related activities. GRC functions are assigned to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ Measures exist to ensure AAT are designed and developed to:
(1) Achieve an appropriate level of accuracy, robustness and cybersecurity;
(2) Perform consistently in those respects throughout the AAT system's lifecycle; and
(3) Be effectively overseen by competent individuals.
Level 4 Quantitatively Controlled
Artificial Intelligence and Autonomous Technology (AAT) capabilities, in addition to being standardized across the entity and centrally managed to ensure consistency across Technology Assets, Applications, Services and/or Data (TAASD), efforts are metrics driven to provide sufficient insight for decision makers to predict optimal performance, ensure continued operations and/or identify areas for improvement. Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Applicable SCR-CMM Level 3 (Well Defined) capabilities are implemented and operational.
▪ Metrics reporting includes quantitative analysis of Key Performance Indicators (KPIs).
▪ Metrics reporting includes quantitative analysis of Key Risk Indicators (KRIs).
▪ Scope of metrics, KPIs and KRIs covers organization-wide cybersecurity and data protection controls, including functions performed by third-parties.
▪ Organizational leadership maintains a formal process to objectively review and respond to metrics, KPIs and KRIs (e.g., monthly or quarterly review).
▪ Based on metrics analysis, process improvement recommendations are submitted for review and are handled in accordance with change control processes.
▪ Business and technical stakeholders are involved in reviewing and approving proposed changes to evolve capabilities.
Level 5 Continuously Improving
Artificial Intelligence and Autonomous Technology (AAT) capabilities are "world class" efforts the leverage predictive analysis (e.g., machine learning, AI, etc.) to enable continuously improving capabilities. Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Applicable SCR-CMM Level 3 (Well Defined) capabilities are implemented and operational.
▪ Based on predictive analysis, process improvements are implemented according to “continuous improvement” practices that affect process changes.
▪ Stakeholders make time-sensitive decisions to support operational efficiency, which may include automated remediation actions.
1. Overview
| Summary |
Standard |
|
AI & Autonomous Technologies Transparency
|
Description
Mechanisms exist to ensure Artificial Intelligence (AI) and Autonomous Technologies (AAT) are designed and developed so its operation is sufficiently transparent such that output can be easily interpreted by personnel implementing the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Document how AI decisions are made for key tools
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Transparency requirements in AI tool selection
∙ Basic explainability documentation
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ AI explainability policy
∙ Require documentation of AI decision logic
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI transparency program
∙ Explainability requirements in AI development standards
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI explainability framework
∙ XAI tools (e.g., SHAP, LIME)
∙ Model documentation standards
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include:
▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation.
▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control, risk assessments, etc.).
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to ensure AAT are designed and developed so its operation is sufficiently transparent such that output can be easily interpreted by personnel implementing the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
AI & Autonomous Technologies Implementation Documentation
|
Description
Mechanisms exist to ensure Artificial Intelligence (AI) and Autonomous Technologies (AAT) include clear and concise documentation that is relevant, accessible and comprehensible to personnel implementing and maintaining the AAT that, at a minimum, provides:
(1) Contact details of the provider;
(2) Characteristics, capabilities and limitations of performance of the AAT;
(3) Errata from the AAT's initial conformity assessment;
(4) Details necessary to interpret the outputs of the AAT;
(5) Human oversight measures necessary to facilitate the interpretation of the outputs of the AAT;
(6) Computational and hardware resources needed to operate the AAT;
(7) Projected useable lifetime of the AAT; and
(8) A description of the mechanisms included within the AAT system to properly collect, store and interpret event logs.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Document AI implementation steps and configuration
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ AI implementation documentation template
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI implementation documentation policy
∙ Technical documentation requirements
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI documentation program
∙ Standardized implementation documentation templates
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI documentation platform
∙ Automated documentation generation from MLOps tools
∙ Model cards and system cards
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include:
▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation.
▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control, risk assessments, etc.).
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to ensure AAT include clear and concise documentation that is relevant, accessible and comprehensible to personnel implementing and maintaining the AAT that, at a minimum, provides:
(1) Contact details of the provider;
(2) Characteristics, capabilities and limitations of performance of the AAT;
(3) Errata from the AAT's initial conformity assessment;
(4) Details necessary to interpret the outputs of the AAT;
(5) Human oversight measures necessary to facilitate the interpretation of the outputs of the AAT;
(6) Computational and hardware resources needed to operate the AAT;
(7) Projected useable lifetime of the AAT; and
(8) A description of the mechanisms included within the AAT system to properly collect, store and interpret event logs.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
AI & Autonomous Technologies Human Domain Knowledge Reliance
|
Description
Mechanisms exist to document the extent to which human domain knowledge is employed to improve Artificial Intelligence (AI) and Autonomous Technologies (AAT) performance including:
(1) Reinforcement Learning from Human Feedback (RLHF);
(2) Fine-tuning;
(3) Retrieval- augmented generation;
(4) Content moderation; and
(5) Business rules.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Document how human expertise is used to improve AI tools
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Document human feedback mechanisms for AI improvement
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal policy on human domain knowledge use in AI training
∙ RLHF documentation
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI development standards covering human knowledge integration
∙ Documentation requirements
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI training governance framework
∙ RLHF program documentation
∙ Human feedback loop management
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to document the extent to which human domain knowledge is employed to improve AAT performance including:
(1) Reinforcement Learning from Human Feedback (RLHF);
(2) Fine-tuning;
(3) Retrieval- augmented generation;
(4) Content moderation; and
(5) Business rules.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
1.1 References
1.2 Identified Requirements
1.3 Related Regulations
2. Identified Requirements
Requirements
| Source |
Requirement |
3. Related Regulations
Regulations
| Source |
Regulation |
|
EULAW
|
Article 13 Transparency and provision of information to deployers
Article 13
Transparency and provision of information to deployers
1. High-risk AI systems shall be designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret a system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured with a view to achieving compliance with the relevant obligations of the provider and deployer set out in Section 3.
2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to deployers.
3. The instructions for use shall contain at least the following information:
|
(a)
|
the identity and the contact details of the provider and, where applicable, of its authorised representative;
|
|
(b)
|
the characteristics, capabilities and limitations of performance of the high-risk AI system, including:
|
(i)
|
its intended purpose;
|
|
(ii)
|
the level of accuracy, including its metrics, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;
|
|
(iii)
|
any known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety or fundamental rights referred to in Article 9(2);
|
|
(iv)
|
where applicable, the technical capabilities and characteristics of the high-risk AI system to provide information that is relevant to explain its output;
|
|
(v)
|
when appropriate, its performance regarding specific persons or groups of persons on which the system is intended to be used;
|
|
(vi)
|
when appropriate, specifications for the input data, or any other relevant information in terms of the training, validation and testing data sets used, taking into account the intended purpose of the high-risk AI system;
|
|
(vii)
|
where applicable, information to enable deployers to interpret the output of the high-risk AI system and use it appropriately;
|
|
|
(c)
|
the changes to the high-risk AI system and its performance which have been pre-determined by the provider at the moment of the initial conformity assessment, if any;
|
|
(d)
|
the human oversight measures referred to in Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of the high-risk AI systems by the deployers;
|
|
(e)
|
the computational and hardware resources needed, the expected lifetime of the high-risk AI system and any necessary maintenance and care measures, including their frequency, to ensure the proper functioning of that AI system, including as regards software updates;
|
|
(f)
|
where relevant, a description of the mechanisms included within the high-risk AI system that allows deployers to properly collect, store and interpret the logs in accordance with Article 12.
|
|
|
EULAW
|
Article 14 Human oversight
Article 14
1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which they are in use.
2. Human oversight shall aim to prevent or minimise the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular where such risks persist despite the application of other requirements set out in this Section.
3. The oversight measures shall be commensurate with the risks, level of autonomy and context of use of the high-risk AI system, and shall be ensured through either one or both of the following types of measures:
|
(a)
|
measures identified and built, when technically feasible, into the high-risk AI system by the provider before it is placed on the market or put into service;
|
|
(b)
|
measures identified by the provider before placing the high-risk AI system on the market or putting it into service and that are appropriate to be implemented by the deployer.
|
4. For the purpose of implementing paragraphs 1, 2 and 3, the high-risk AI system shall be provided to the deployer in such a way that natural persons to whom human oversight is assigned are enabled, as appropriate and proportionate:
|
(a)
|
to properly understand the relevant capacities and limitations of the high-risk AI system and be able to duly monitor its operation, including in view of detecting and addressing anomalies, dysfunctions and unexpected performance;
|
|
(b)
|
to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias), in particular for high-risk AI systems used to provide information or recommendations for decisions to be taken by natural persons;
|
|
(c)
|
to correctly interpret the high-risk AI system’s output, taking into account, for example, the interpretation tools and methods available;
|
|
(d)
|
to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output of the high-risk AI system;
|
|
(e)
|
to intervene in the operation of the high-risk AI system or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.
|
5. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 of this Article shall be such as to ensure that, in addition, no action or decision is taken by the deployer on the basis of the identification resulting from the system unless that identification has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority.
The requirement for a separate verification by at least two natural persons shall not apply to high-risk AI systems used for the purposes of law enforcement, migration, border control or asylum, where Union or national law considers the application of this requirement to be disproportionate.
|
|
EULAW
|
Article 15 Accuracy, robustness and cybersecurity
Article 15
Accuracy, robustness and cybersecurity
1. High-risk AI systems shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle.
2. To address the technical aspects of how to measure the appropriate levels of accuracy and robustness set out in paragraph 1 and any other relevant performance metrics, the Commission shall, in cooperation with relevant stakeholders and organisations such as metrology and benchmarking authorities, encourage, as appropriate, the development of benchmarks and measurement methodologies.
3. The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use.
4. High-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems. Technical and organisational measures shall be taken in this regard.
The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans.
High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way as to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and as to ensure that any such feedback loops are duly addressed with appropriate mitigation measures.
5. High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities.
The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks.
The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.
|
|
EULAW
|
Article 16 Obligations of providers of high-risk AI systems
Article 16
Obligations of providers of high-risk AI systems
Providers of high-risk AI systems shall:
|
(a)
|
ensure that their high-risk AI systems are compliant with the requirements set out in Section 2;
|
|
(b)
|
indicate on the high-risk AI system or, where that is not possible, on its packaging or its accompanying documentation, as applicable, their name, registered trade name or registered trade mark, the address at which they can be contacted;
|
|
(c)
|
have a quality management system in place which complies with Article 17;
|
|
(d)
|
keep the documentation referred to in Article 18;
|
|
(e)
|
when under their control, keep the logs automatically generated by their high-risk AI systems as referred to in Article 19;
|
|
(f)
|
ensure that the high-risk AI system undergoes the relevant conformity assessment procedure as referred to in Article 43, prior to its being placed on the market or put into service;
|
|
(g)
|
draw up an EU declaration of conformity in accordance with Article 47;
|
|
(h)
|
affix the CE marking to the high-risk AI system or, where that is not possible, on its packaging or its accompanying documentation, to indicate conformity with this Regulation, in accordance with Article 48;
|
|
(i)
|
comply with the registration obligations referred to in Article 49(1);
|
|
(j)
|
take the necessary corrective actions and provide information as required in Article 20;
|
|
(k)
|
upon a reasoned request of a national competent authority, demonstrate the conformity of the high-risk AI system with the requirements set out in Section 2;
|
|
(l)
|
ensure that the high-risk AI system complies with accessibility requirements in accordance with Directives (EU) 2016/2102 and (EU) 2019/882.
|
|
|
EULAW
|
Article 17 Quality management system
Article 17
Quality management system
1. Providers of high-risk AI systems shall put a quality management system in place that ensures compliance with this Regulation. That system shall be documented in a systematic and orderly manner in the form of written policies, procedures and instructions, and shall include at least the following aspects:
|
(a)
|
a strategy for regulatory compliance, including compliance with conformity assessment procedures and procedures for the management of modifications to the high-risk AI system;
|
|
(b)
|
techniques, procedures and systematic actions to be used for the design, design control and design verification of the high-risk AI system;
|
|
(c)
|
techniques, procedures and systematic actions to be used for the development, quality control and quality assurance of the high-risk AI system;
|
|
(d)
|
examination, test and validation procedures to be carried out before, during and after the development of the high-risk AI system, and the frequency with which they have to be carried out;
|
|
(e)
|
technical specifications, including standards, to be applied and, where the relevant harmonised standards are not applied in full or do not cover all of the relevant requirements set out in Section 2, the means to be used to ensure that the high-risk AI system complies with those requirements;
|
|
(f)
|
systems and procedures for data management, including data acquisition, data collection, data analysis, data labelling, data storage, data filtration, data mining, data aggregation, data retention and any other operation regarding the data that is performed before and for the purpose of the placing on the market or the putting into service of high-risk AI systems;
|
|
(g)
|
the risk management system referred to in Article 9;
|
|
(h)
|
the setting-up, implementation and maintenance of a post-market monitoring system, in accordance with Article 72;
|
|
(i)
|
procedures related to the reporting of a serious incident in accordance with Article 73;
|
|
(j)
|
the handling of communication with national competent authorities, other relevant authorities, including those providing or supporting the access to data, notified bodies, other operators, customers or other interested parties;
|
|
(k)
|
systems and procedures for record-keeping of all relevant documentation and information;
|
|
(l)
|
resource management, including security-of-supply related measures;
|
|
(m)
|
an accountability framework setting out the responsibilities of the management and other staff with regard to all the aspects listed in this paragraph.
|
2. The implementation of the aspects referred to in paragraph 1 shall be proportionate to the size of the provider’s organisation. Providers shall, in any event, respect the degree of rigour and the level of protection required to ensure the compliance of their high-risk AI systems with this Regulation.
3. Providers of high-risk AI systems that are subject to obligations regarding quality management systems or an equivalent function under relevant sectoral Union law may include the aspects listed in paragraph 1 as part of the quality management systems pursuant to that law.
4. For providers that are financial institutions subject to requirements regarding their internal governance, arrangements or processes under Union financial services law, the obligation to put in place a quality management system, with the exception of paragraph 1, points (g), (h) and (i) of this Article, shall be deemed to be fulfilled by complying with the rules on internal governance arrangements or processes pursuant to the relevant Union financial services law. To that end, any harmonised standards referred to in Article 40 shall be taken into account.
|
Linked Issues
- Secure Controls Framework -
"The SCF is the Common Controls Framework™ (CCF), the world's most comprehensive cybersecurity and data privacy metaframework - it is also free to use. The entire concept is building secure, compliant and resilient capabilities in the most efficient and cost-effective manner possible.
The SCF is more than just a unified control catalog, since its included content creates a playbook for Governance, Risk & Compliance (GRC) capabilities. Used globally by organizations of every size, the SCF is a robust and scalable solution for security, compliance and resilience controls. As a comprehensive security framework, the SCF maps 1,400+ controls across 200+ laws, regulations, and industry frameworks so you can implement once and comply everywhere.
Like it or not, cybersecurity is a protracted war on an asymmetric battlefield, where the threats are everywhere and as defenders we have to make the effort to work together to help improve cybersecurity and data privacy practices, since we all suffer when massive data breaches occur or when cyber attacks have physical impacts. Hackers share information on attack methods with other hackers, so why shouldn’t the good guys share information on how to best protect an organization? We decided to take action and make a difference, since we feel it is too important to wait for someone else to fix the problems that exist.
The SCF is made up of volunteers, mainly specialists within the cybersecurity profession, who focus on GRC and the cybersecurity side of data privacy. These are auditors, engineers, architects, incident responders, consultants and other specialists who live and breathe these topics on a daily basis. The end product is "expert-derived content" that makes up the SCF." https://securecontrolsframework.com/
Terms & Conditions
The SCF End User License Agreement (EULA) governs the use of the Secure Controls Framework® (SCF) under the Creative Commons Attribution-No Derivatives 4.0 International Public License.
|