+Artículo 60 Pruebas de sistemas de IA de alto riesgo en condiciones reales fuera de los espacios controlados de pruebas para la IA
---+Artículo 60 1.
---+Artículo 60 2.
---+Artículo 60 3.
---+Artículo 60 4.
---+Artículo 60 5.
---+Artículo 60 6.
---+Artículo 60 7.
---+Artículo 60 8.
---+Artículo 60 9.
|
Artículo 60 Pruebas de sistemas de IA de alto riesgo en condiciones reales fuera de los espacios controlados de pruebas para la IA
-
1. Resumen
| Título |
Regulación |
|
Artículo 60 1.
|
Los proveedores o proveedores potenciales de sistemas de IA de alto riesgo enumerados en el anexo III o regulados por los actos legislativos de armonización de la Unión enumerados en el anexo I, sección A, podrán realizar pruebas de sistemas de IA de alto riesgo en condiciones reales fuera de los espacios controlados de pruebas para la IA de conformidad con el presente artículo y con el plan de la prueba en condiciones reales a que se refiere el presente artículo, sin perjuicio de las prohibiciones establecidas en el artículo 5. La Comisión adoptará, mediante un acto de ejecución, los elementos detallados del plan de la prueba en condiciones reales. Dichos actos de ejecución se adoptarán de conformidad con el procedimiento de examen a que se refiere el artículo 98, apartado 2. El presente apartado se entiende sin perjuicio del Derecho de la Unión o nacional en materia de pruebas en condiciones reales de sistemas de IA de alto riesgo asociados a productos regulados por los actos legislativos de armonización de la Unión enumerados en el anexo I.
|
|
Artículo 60 2.
|
Los proveedores o proveedores potenciales podrán realizar pruebas de los sistemas de IA de alto riesgo mencionados en el anexo III o regulados por los actos legislativos de armonización de la Unión enumerados en el anexo I, sección A, en condiciones reales en cualquier momento antes de la introducción en el mercado o la puesta en servicio del sistema de IA de alto riesgo por cuenta propia o en asociación con uno o varios responsables del despliegue o responsables del despliegue potenciales.
|
|
Artículo 60 3.
|
Las pruebas de sistemas de IA de alto riesgo en condiciones reales con arreglo al presente artículo se entenderán sin perjuicio de cualquier revisión ética que se exija en el Derecho de la Unión o nacional.
|
|
Artículo 60 4.
|
Los proveedores o proveedores potenciales podrán realizar pruebas en condiciones reales solamente cuando se cumplan todas las condiciones siguientes:
a)
el proveedor o proveedor potencial ha elaborado un plan de la prueba en condiciones reales y lo ha presentado a la autoridad de vigilancia del mercado del Estado miembro en que se vayan a realizar las pruebas en condiciones reales;
b)
la autoridad de vigilancia del mercado del Estado miembro en que se vayan a realizar las pruebas en condiciones reales ha aprobado las pruebas en condiciones reales y el plan de la prueba en condiciones reales; si la autoridad de vigilancia del mercado no responde en un plazo de treinta días, se entenderá que las pruebas en condiciones reales y el plan de la prueba en condiciones reales han sido aprobados; cuando el Derecho nacional no contemple una aprobación tácita, las pruebas en condiciones reales estarán sujetas a una autorización también en este caso;
c)
el proveedor o proveedor potencial, con excepción de los proveedores o proveedores potenciales de sistemas de IA de alto riesgo mencionados en el anexo III, puntos 1, 6 y 7, en los ámbitos de la garantía del cumplimiento del Derecho, la migración, el asilo y la gestión del control fronterizo, así como de los sistemas de IA de alto riesgo mencionados en el punto 2 del anexo III ha registrado las pruebas en condiciones reales de conformidad con el artículo 71, apartado 4, con un número de identificación único para toda la Unión y la información indicada en el anexo IX; el proveedor o proveedor potencial de sistemas de IA de alto riesgo a que se refiere el anexo III, puntos 1, 6 y 7, en los ámbitos de la garantía del cumplimiento del Derecho, la migración, el asilo y la gestión del control fronterizo, ha registrado las pruebas en condiciones reales en la parte no pública de la base de datos de la UE de conformidad con el artículo 49, apartado 4, letra d), con un número de identificación único para toda la Unión y la información indicada en este; el proveedor o proveedor potencial de sistemas de IA de alto riesgo a que se refiere el anexo III, punto 2, ha registrado las pruebas en condiciones reales de conformidad con el artículo 49, apartado 5;
d)
el proveedor o proveedor potencial que realiza las pruebas en condiciones reales está establecido en la Unión o ha designado a un representante legal que está establecido en la Unión;
e)
los datos recabados y tratados a efectos de las pruebas en condiciones reales únicamente se transferirán a terceros países si se aplican las garantías adecuadas y aplicables en virtud del Derecho de la Unión;
f)
las pruebas en condiciones reales no duran más de lo necesario para lograr sus objetivos y, en cualquier caso, no más de seis meses, que podrán prorrogarse por un período adicional de seis meses, con sujeción al envío de una notificación previa por parte del proveedor o proveedor potencial a la autoridad de vigilancia del mercado, acompañada por una explicación de la necesidad de dicha prórroga;
g)
los sujetos de las pruebas en condiciones reales que sean personas pertenecientes a colectivos vulnerables debido a su edad o a una discapacidad cuentan con protección adecuada;
h)
cuando un proveedor o proveedor potencial organice las pruebas en condiciones reales en cooperación con uno o varios responsables del despliegue o responsables del despliegue potenciales, estos últimos habrán sido informados de todos los aspectos de las pruebas que resulten pertinentes para su decisión de participar y habrán recibido las instrucciones de uso pertinentes del sistema de IA a que se refiere el artículo 13; el proveedor o proveedor potencial y el responsable del despliegue o responsable del despliegue potencial alcanzarán un acuerdo en que se detallen sus funciones y responsabilidades con vistas a garantizar el cumplimiento de las disposiciones relativas a las pruebas en condiciones reales con arreglo al presente Reglamento y a otras disposiciones de Derecho de la Unión y nacional aplicable;
i)
los sujetos de las pruebas en condiciones reales han dado su consentimiento informado de conformidad con el artículo 61 o, en el ámbito de la garantía del cumplimiento del Derecho, en el que intentar obtener el consentimiento informado impediría que se probara el sistema de IA, las pruebas en sí y los resultados de las pruebas en condiciones reales no tendrán ningún efecto negativo sobre los sujetos, cuyos datos personales se suprimirán una vez realizada la prueba;
j)
las pruebas en condiciones reales son supervisadas de manera efectiva por el proveedor o el proveedor potencial y por los responsables del despliegue o los responsables del despliegue potenciales mediante personas adecuadamente cualificadas en el ámbito pertinente y con la capacidad, formación y autoridad necesarias para realizar sus tareas;
k)
se pueden revertir y descartar de manera efectiva las predicciones, recomendaciones o decisiones del sistema de IA.
|
|
Artículo 60 5.
|
Cualquier sujeto de las pruebas en condiciones reales o su representante legalmente designado, según proceda, podrá, sin sufrir por ello perjuicio alguno y sin tener que proporcionar ninguna justificación, abandonar las pruebas en cualquier momento retirando su consentimiento informado y solicitar la supresión inmediata y permanente de sus datos personales. La retirada del consentimiento informado no afectará a las actividades ya completadas.
|
|
Artículo 60 6.
|
De conformidad con el artículo 75, los Estados miembros conferirán a sus autoridades de vigilancia del mercado poderes para exigir a los proveedores y proveedores potenciales que faciliten información, realizar sin previo aviso inspecciones a distancia o in situ y controlar la realización de las pruebas en condiciones reales y los sistemas de IA de alto riesgo relacionados. Las autoridades de vigilancia del mercado harán uso de dichos poderes para garantizar que las pruebas en condiciones reales se desarrollen de manera segura.
|
|
Artículo 60 7.
|
Se informará de cualquier incidente grave detectado en el transcurso de las pruebas en condiciones reales a la autoridad nacional de vigilancia del mercado de conformidad con el artículo 73. El proveedor o proveedor potencial adoptará medidas de reducción inmediatas o, en su defecto, suspenderá las pruebas en condiciones reales hasta que se produzca dicha reducción o pondrá fin a las pruebas. El proveedor o proveedor potencial establecerá un procedimiento para la rápida recuperación del sistema de IA en caso de que se ponga fin a las pruebas en condiciones reales.
|
|
Artículo 60 8.
|
El proveedor o proveedor potencial notificará a la autoridad nacional de vigilancia del mercado del Estado miembro en que se vayan a realizar las pruebas en condiciones reales la suspensión o la terminación de las pruebas en condiciones reales y los resultados finales.
|
|
Artículo 60 9.
|
El proveedor o proveedor potencial será responsable, conforme al Derecho de la Unión y nacional en materia de responsabilidad aplicable, de cualquier daño causado en el transcurso de sus pruebas en condiciones reales.
|
,
1.1 Referencias
1.2 Requisitos identificados
1.3 Estándares relacionados
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- AI TEVV Reporting (SCF)
- AI TEVV Reporting (SCF)
- Serious Incident Reporting For AI & Autonomous Technologies (SCF)
2. Requisitos identificados
Requisitos
| Fuente |
Requisito |
3. Estándares relacionados
Estándares
| Fuente |
Requisito |
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
AI TEVV Reporting
Description
Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report AI testing results to management
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Document and communicate AI test results to stakeholders
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI TEVV reporting process ∙ Standardized reporting templates
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI TEVV reporting program ∙ Regular stakeholder briefings ∙ Findings tracking
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI TEVV reporting framework ∙ Automated test result dashboards ∙ Board-level AI risk reporting
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
AI TEVV Reporting
Description
Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report AI testing results to management
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Document and communicate AI test results to stakeholders
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI TEVV reporting process ∙ Standardized reporting templates
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI TEVV reporting program ∙ Regular stakeholder briefings ∙ Findings tracking
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI TEVV reporting framework ∙ Automated test result dashboards ∙ Board-level AI risk reporting
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Serious Incident Reporting For AI & Autonomous Technologies
Description
Mechanisms exist to report any serious incident involving operational Artificial Intelligence (AI) and Autonomous Technologies (AAT) to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report serious AI incidents to relevant authorities as required
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Incident reporting procedure for serious AI incidents
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal serious AI incident reporting process ∙ Regulatory reporting contacts
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI incident response plan with regulatory reporting requirements ∙ Legal team involvement
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI incident response program ∙ Automated regulatory notification workflows ∙ Legal and compliance integration
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include: ▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation. ▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control and risk assessments). ▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners. ▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD). ▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards and procedures & guidelines). ▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report any serious incident involving operational AAT to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|