+Článok 60 Testovanie vysokorizikových systémov AI v reálnych podmienkach mimo regulačných experimentálnych prostredí pre AI
---+Článok 60 1.
---+Článok 60 2.
---+Článok 60 3.
---+Článok 60 4.
---+Článok 60 5.
---+Článok 60 6.
---+Článok 60 7.
---+Článok 60 8.
---+Článok 60 9.

Článok 60 Testovanie vysokorizikových systémov AI v reálnych podmienkach mimo regulačných experimentálnych prostredí pre AI

-

1. Prehľad

,
Článok 60 1.
Poskytovatelia alebo potenciálni poskytovatelia vysokorizikových systémov AI, ktoré sú uvedené v prílohe III alebo na ktoré sa vzťahujú harmonizačné právne predpisy Únie uvedené v oddiele A prílohy I, môžu vykonávať testovanie vysokorizikových systémov AI v reálnych podmienkach mimo regulačných experimentálnych prostredí pre AI v súlade s týmto článkom a plánom testovania v reálnych podmienkach uvedeným v tomto článku bez toho, aby boli dotknuté zákazy podľa článku 5.

Komisia prostredníctvom vykonávacích aktov stanoví podrobné prvky plánu testovania v reálnych podmienkach. Uvedené vykonávacie akty sa prijmú v súlade s postupom preskúmania uvedeným v článku 98 ods. 2.

Týmto odsekom nie je dotknuté právo Únie ani vnútroštátne právo týkajúce sa testovania v reálnych podmienkach, pokiaľ ide o vysokorizikové systémy AI súvisiace s výrobkami, na ktoré sa vzťahujú harmonizačné právne predpisy Únie uvedené v prílohe I.
Článok 60 2.
Poskytovatelia alebo potenciálni poskytovatelia môžu vykonávať testovanie vysokorizikových systémov AI, ktoré sú uvedené v prílohe III alebo na ktoré sa vzťahujú harmonizačné právne predpisy Únie uvedené v oddiele A prílohy I, v reálnych podmienkach kedykoľvek pred uvedením vysokorizikového systému AI na trh alebo do prevádzky samostatne alebo v partnerstve s jedným alebo viacerými nasadzujúcimi subjektmi alebo potenciálnymi nasadzujúcimi subjektmi.
Článok 60 3.
Testovaním vysokorizikových systémov AI v reálnych podmienkach podľa tohto článku nie je dotknuté etické preskúmanie, ktoré sa vyžaduje podľa práva Únie alebo vnútroštátneho práva.
Článok 60 4.

Poskytovatelia alebo potenciálni poskytovatelia môžu vykonávať testovanie v reálnych podmienkach, len ak sú splnené všetky tieto podmienky:

a) 

poskytovateľ alebo potenciálny poskytovateľ vypracoval plán testovania v reálnych podmienkach a predložil ho orgánu dohľadu nad trhom v členskom štáte, v ktorom sa má testovanie v reálnych podmienkach vykonať;

b) 

orgán dohľadu nad trhom v členskom štáte, v ktorom sa má testovanie v reálnych podmienkach vykonať, schválil testovanie v reálnych podmienkach a plán testovania v reálnych podmienkach; ak orgán dohľadu nad trhom neposkytol odpoveď do 30 dní, testovanie v reálnych podmienkach a plán testovania v reálnych podmienkach sa považujú za schválené; ak sa vo vnútroštátnom práve nestanovuje konkludentný súhlas, testovanie v reálnych podmienkach podlieha povoleniu;

c) 

poskytovateľ alebo potenciálny poskytovateľ s výnimkou poskytovateľov alebo potenciálnych poskytovateľov vysokorizikových systémov AI uvedených v prílohe III bodoch 1, 6 a 7 v oblastiach presadzovania práva, migrácie, azylu a riadenia kontroly hraníc a vysokorizikových systémov AI uvedených v prílohe III bode 2 zaregistroval testovanie v reálnych podmienkach v súlade s článkom 71 ods. 4 s jedinečným jednotným identifikačným číslom pre celú Úniu a s informáciami uvedenými v prílohe IX; poskytovateľ alebo potenciálny poskytovateľ vysokorizikových systémov AI uvedených v prílohe III bodoch 1, 6 a 7 v oblastiach presadzovania práva, migrácie, azylu a riadenia kontroly hraníc zaregistroval testovanie v reálnych podmienkach v neverejnej časti databázy Únie podľa článku 49 ods. 4 písm. d) s jedinečným jednotným identifikačným číslom pre celú Úniu a s informáciami v ňom uvedenými; poskytovateľ alebo potenciálny poskytovateľ vysokorizikových systémov AI uvedených v prílohe III bode 2 zaregistroval testovanie v reálnych podmienkach v súlade s článkom 49 ods. 5;

d) 

poskytovateľ alebo potenciálny poskytovateľ vykonávajúci testovanie v reálnych podmienkach je usadený v Únii alebo vymenoval právneho zástupcu, ktorý je usadený v Únii;

e) 

údaje zozbierané a spracúvané na účely testovania v reálnych podmienkach sa prenášajú do tretích krajín len za predpokladu, že sa implementujú primerané a uplatniteľné záruky podľa práva Únie;

f) 

testovanie v reálnych podmienkach netrvá dlhšie, ako je potrebné na dosiahnutie jeho cieľov, a v žiadnom prípade nie dlhšie ako šesť mesiacov, ktoré možno predĺžiť o ďalších šesť mesiacov, ak o tom poskytovateľ alebo potenciálny poskytovateľ vopred informoval orgán dohľadu nad trhom spolu s vysvetlením potreby takéhoto predĺženia;

g) 

účastníci testovania v reálnych podmienkach, ktorí sú osobami patriacimi do zraniteľných skupín z dôvodu ich veku alebo zdravotného postihnutia, sú primerane chránení;

h) 

ak poskytovateľ alebo potenciálny poskytovateľ organizuje testovanie v reálnych podmienkach v spolupráci s jedným alebo viacerými nasadzujúcimi subjektmi alebo potenciálnymi nasadzujúcimi subjektmi, poskytnú sa takýmto subjektom informácie o všetkých aspektoch testovania, ktoré sú relevantné pre ich rozhodnutie zúčastniť sa, ako aj príslušný návod na použitie systému AI uvedený v článku 13; poskytovateľ alebo potenciálny poskytovateľ a nasadzujúci subjekt alebo potenciálny nasadzujúci subjekt uzavrú dohodu, v ktorej sa stanovia ich úlohy a povinnosti s cieľom zabezpečiť súlad s ustanoveniami o testovaní v reálnych podmienkach podľa tohto nariadenia a iného uplatniteľného práva Únie a vnútroštátneho práva;

i) 

účastníci testovania v reálnych podmienkach poskytli informovaný súhlas v súlade s článkom 61 alebo, v prípade presadzovania práva, ak by získanie informovaného súhlasu bránilo testovaniu systému AI, samotné testovanie a výsledok testovania v reálnych podmienkach nesmú mať žiaden negatívny vplyv na účastníkov a osobné údaje účastníkov sa po vykonaní testovania vymažú;

j) 

na testovanie v reálnych podmienkach účinne dohliada poskytovateľ alebo potenciálny poskytovateľ a nasadzujúce subjekty alebo potenciálne nasadzujúce subjekty prostredníctvom osôb, ktoré sú primerane kvalifikované v príslušnej oblasti a majú potrebnú kapacitu, odbornú prípravu a právomoc na vykonávanie svojich úloh;

k) 

predpovede, odporúčania alebo rozhodnutia systému AI možno účinne zvrátiť a nezohľadniť.
Článok 60 5.
Ktorýkoľvek účastník testovania v reálnych podmienkach alebo v relevantnom prípade jeho zákonne určený zástupca, sa môže kedykoľvek stiahnuť z testovania odvolaním svojho informovaného súhlasu bez akejkoľvek následnej ujmy a bez toho, aby musel poskytnúť akékoľvek odôvodnenie, a požiadať o bezodkladné a trvalé vymazanie svojich osobných údajov. Odvolanie informovaného súhlasu nemá vplyv na činnosti, ktoré už boli vykonané.
Článok 60 6.
V súlade s článkom 75 členské štáty prenesú na svoje orgány dohľadu nad trhom právomoc požadovať informácie od poskytovateľov a potenciálnych poskytovateľov, vykonávať neohlásené kontroly na diaľku alebo na mieste a vykonávať kontroly vykonávania testovania v reálnych podmienkach a súvisiacich vysokorizikových systémov AI. Orgány dohľadu nad trhom využívajú túto právomoc na zaistenie bezpečného vývoja testovania v reálnych podmienkach.
Článok 60 7.
Každý závažný incident zistený počas testovania v reálnych podmienkach sa oznamuje vnútroštátnemu orgánu dohľadu nad trhom v súlade s článkom 73. Poskytovateľ alebo potenciálny poskytovateľ prijme okamžité opatrenia na zmiernenie dôsledkov incidentu, alebo ak to nie je možné, pozastaví testovanie v reálnych podmienkach dovtedy, kým nedôjde k takémuto zmierneniu, alebo ho v opačnom prípade ukončí. Poskytovateľ alebo potenciálny poskytovateľ stanoví postup rýchleho stiahnutia systému AI od používateľa pri takomto ukončení testovania v reálnych podmienkach.
Článok 60 8.
Poskytovatelia alebo potenciálni poskytovatelia informujú vnútroštátny orgán dohľadu nad trhom v členskom štáte, v ktorom sa vykonáva testovanie v reálnych podmienkach, o pozastavení alebo ukončení testovania v reálnych podmienkach a o konečných výsledkoch.
Článok 60 9.
Poskytovateľ alebo potenciálny poskytovateľ je zodpovedný za všetky škody, ktoré vzniknú počas testovania v reálnych podmienkach, podľa uplatniteľného práva Únie a vnútroštátneho práva v oblasti zodpovednosti.

1.1 Odkazy

1.2 Identifikované požiadavky

1.3 Súvisiace štandardy

2. Identifikované požiadavky

Požiadavky

3. Súvisiace štandardy

Štandardy
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)

Description

Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Information Assurance (IA) Program
∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security)
∙ AI governance program

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Information Assurance (IA) Program
∙ Formal AI TEVV process for AI systems
∙ AI governance program

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Information Assurance (IA) Program
∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function
∙ AI testing tools (e.g., IBM OpenScale, Great Expectations)
∙ AI governance program

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program
∙ NIST AI RMF Measure function
∙ Third-party AI testing for high-risk systems
∙ AI testing integrated with CI/CD pipelines

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Information Assurance (IA) Program
∙ Enterprise AI TEVV program (NIST AI RMF Measure function)
∙ Independent AI testing and evaluation for high-risk systems
∙ AI TEVV integrated with MLOps and CI/CD
∙ EU AI Act conformity assessment (if applicable)

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF AI TEVV Reporting

Description

Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Report AI testing results to management

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Document and communicate AI test results to stakeholders

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Formal AI TEVV reporting process
∙ Standardized reporting templates

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ AI TEVV reporting program
∙ Regular stakeholder briefings
∙ Findings tracking

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Enterprise AI TEVV reporting framework
∙ Automated test result dashboards
∙ Board-level AI risk reporting

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF AI TEVV Reporting

Description

Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Report AI testing results to management

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Document and communicate AI test results to stakeholders

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Formal AI TEVV reporting process
∙ Standardized reporting templates

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ AI TEVV reporting program
∙ Regular stakeholder briefings
∙ Findings tracking

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Enterprise AI TEVV reporting framework
∙ Automated test result dashboards
∙ Board-level AI risk reporting

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.

Level 2 Planned Tracked

SCR-CMM Level 2 criteria definitions are not available for this control:
▪ A reasonable person would conclude a well-defined and standardized process is required.
▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization.
▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
SCF Serious Incident Reporting For AI & Autonomous Technologies

Description

Mechanisms exist to report any serious incident involving operational Artificial Intelligence (AI) and Autonomous Technologies (AAT) to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.

Possible Solutions & Considerations

Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2

∙ Report serious AI incidents to relevant authorities as required

Small Business (10-49 staff) / BLS Firm Size Classes 3-4

∙ Incident reporting procedure for serious AI incidents

Medium Business (50-249 staff) / BLS Firm Size Classes 5-6

∙ Formal serious AI incident reporting process
∙ Regulatory reporting contacts

Large Business (250-999 staff) / BLS Firm Size Classes 7-8

∙ AI incident response plan with regulatory reporting requirements
∙ Legal team involvement

Enterprise (> 1,000 staff) / BLS Firm Size Class 9

∙ Enterprise AI incident response program
∙ Automated regulatory notification workflows
∙ Legal and compliance integration

SCR-CMM

Level 0 Not Performed

Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.

Level 1 Performed Informally

Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include:
▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation.
▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control and risk assessments).
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.

Level 2 Planned Tracked

Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards and procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.

Level 3 Well Defined

Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to report any serious incident involving operational AAT to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.

Level 4 Quantitatively Controlled

Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.

Level 5 Continuously Improving

Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.