+Airteagal 60 Córais intleachta saorga ardriosca a thástáil i bhfíordhálaí lasmuigh de bhoscaí gainimh rialála intleachta saorga
---+Airteagal 60 1.
---+Airteagal 60 2.
---+Airteagal 60 3.
---+Airteagal 60 4.
---+Airteagal 60 5.
---+Airteagal 60 6.
---+Airteagal 60 7.
---+Airteagal 60 8.
---+Airteagal 60 9.
|
Airteagal 60 Córais intleachta saorga ardriosca a thástáil i bhfíordhálaí lasmuigh de bhoscaí gainimh rialála intleachta saorga
-
1. Forbhreathnú
| Teideal |
Rialáil |
|
Airteagal 60 1.
|
Féadfaidh soláthraithe nó soláthraithe ionchasacha córas intleachta saorga ardriosca a liostaítear in Iarscríbhinn III nó a chumhdaítear le reachtaíocht chomhchuibhithe an Aontais a liostaítear i Roinn A de Iarscríbhinn I, tástáil a dhéanamh ar chórais intleachta saorga ardriosca i bhfíordhálaí lasmuigh de bhoscaí gainimh rialála intleachta saorga, i gcomhréir leis an Airteagal seo agus leis an bplean tástála i bhfíordhálaí dá dtagraítear san Airteagal seo, gan dochar do na toirmisc faoi Airteagal 5. Déanfaidh an Coimisiún, trí bhíthin gníomhartha cur chun feidhme, gnéithe mionsonraithe an phlean chun tástáil a dhéanamh i bhfíordhálaí a shonrú. Déanfar na gníomhartha cur chun feidhme sin a ghlacadh i gcomhréir leis an nós imeachta scrúdúcháin dá dtagraítear in Airteagal 98(2). Beidh an mhír seo gan dochar do dhlí an Aontais nó don dlí náisiúnta maidir le córais intleachta saorga ardriosca a bhaineann le táirgí a chumhdaítear le reachtaíocht chomhchuibhithe an Aontais a liostaítear in Iarscríbhinn I a thástáil i bhfíordhálaí.
|
|
Airteagal 60 2.
|
Féadfaidh soláthraithe nó soláthraithe ionchasacha tástáil a dhéanamh ar na córais intleachta saorga ardriosca dá dtagraítear in Iarscríbhinn III nó a chumhdaítear le reachtaíocht chomhchuibhithe an Aontais a liostaítear i Roinn A de Iarscríbhinn I i bhfíordhálaí tráth ar bith sula gcuirfear an córas intleachta saorga ardriosca ar an margadh nó sula gcuirfear i mbun seirbhíse é ar a gconlán féin nó i gcomhpháirtíocht le húsáideoir gairmiúil nó úsáideoir gairmiúil ionchasach amháin nó níos mó.
|
|
Airteagal 60 3.
|
Beidh tástáil córas intleachta saorga ardriosca i bhfíordhálaí faoin Airteagal seo gan dochar d’aon athbhreithniú eiticiúil a cheanglaítear le dlí an Aontais nó leis an dlí náisiúnta.
|
|
Airteagal 60 4.
|
Ní fhéadfaidh soláthraithe nó soláthraithe ionchasacha an tástáil a dhéanamh i bhfíordhálaí ach amháin i gcás ina gcomhlíontar gach ceann de na coinníollacha seo a leanas:
(a)
tá plean tástála i bhfíordhálaí tarraingthe suas ag an soláthraí nó ag an soláthraí ionchasach agus é curtha faoi bhráid an údaráis faireachais margaidh sa Bhallstát ina bhfuil an tástáil i bhfíordhálaí le déanamh;
(b)
tá an tástáil i bhfíordhálaí agus plean na tástála i bhfíordhálaí formheasta ag an údarás faireachais margaidh sa Bhallstát ina bhfuil an tástáil le déanamh i bhfíordhálaí; i gcás nár thug an t-údarás faireachais margaidh freagra laistigh de 30 lá, tuigfear go bhfuil an tástáil i bhfíordhálaí agus plean na tástála i bhfíordhálaí formheasta; i gcás nach ndéantar foráil sa dlí náisiúnta maidir le formheas intuigthe, leanfaidh an tástáil i bhfíordhálaí de bheith faoi réir údaraithe;
(c)
chláraigh an soláthraí nó an soláthraí ionchasach, cé is moite de sholáthraithe nó soláthraithe ionchasacha córas intleachta saorga ardriosca dá dtagraítear i bpointí 1, 6 agus 7 d’Iarscríbhinn III i réimsí fhorfheidhmiú an dlí, na himirce, an bhainistithe tearmainn agus an bhainistithe rialaithe teorann, agus na gcóras intleachta saorga ardriosca dá dtagraítear i bpointe 2 d’Iarscríbhinn III an tástáil i bhfíordhálaí i gcomhréir le hAirteagal 71(4) le huimhir aitheantais uathúil aonair uile-Aontais agus leis an bhfaisnéis a shonraítear in Iarscríbhinn IX; chláraigh soláthraí nó soláthraí ionchasach na gcóras intleachta saorga ardriosca dá dtagraítear i bpointí 1, 6 agus 7 d’Iarscríbhinn III i réimsí fhorfheidhmiú an dlí, na himirce, an tearmainn agus an bhainistithe rialaithe teorann, an tástáil i bhfíordhálaí sa roinn shlán neamhphoiblí de bhunachar sonraí an Aontais de réir Airteagal 49(4), pointe (d), le huimhir aitheantais uathúil aonair uile-Aontais agus leis an bhfaisnéis a shonraítear ann; gur chláraigh soláthraí nó soláthraí ionchasach na gcóras intleachta saorga ardriosca dá dtagraítear i bpointe 2 d’Iarscríbhinn III an tástáil i bhfíordhálaí i gcomhréir le hAirteagal 49(5);
(d)
tá an soláthraí nó an soláthraí ionchasach atá ag déanamh na tástála i bhfíordhálaí bunaithe san Aontas nó tá ionadaí dlíthiúil ceaptha aige atá bunaithe san Aontas;
(e)
ní dhéanfar sonraí a bhailítear agus a phróiseáiltear chun críche tástála i bhfíordhálaí a aistriú chuig tríú tíortha ach amháin ar choinníoll go gcuirfear coimircí iomchuí is infheidhme faoi dhlí an Aontais chun feidhme;
(f)
ní mhaireann an tástáil i bhfíordhálaí níos faide ná mar is gá chun a cuspóirí a bhaint amach, agus in aon chás ní mhairfidh sí níos faide ná 6 mhí, a fhéadfar a fhadú ar feadh tréimhse bhreise 6 mhí, faoi réir fógra a fháil roimh ré ón soláthraí nó soláthraí ionchasach don údarás faireachais margaidh, mar aon le míniú ar an ngá atá leis an bhfadú sin;
(g)
tabharfar cosaint iomchuí do dhaoine is ábhar don tástáil i bhfíordhálaí ar daoine ar de ghrúpaí leochaileacha iad mar gheall ar a n-aois nó a míchumas;
(h)
i gcás ina n-eagraíonn soláthraí nó soláthraí ionchasach an tástáil i bhfíordhálaí i gcomhar le húsáideoir gairmiúil ionchasach amháin nó níos mó úsáideoirí gairmiúla ionchasacha, cuireadh an t-úsáideoir gairmiúil ionchasach sin nó na húsáideoirí ionchasacha sin ar an eolas faoi gach gné den tástáil atá ábhartha dá gcinneadh a bheith rannpháirteach, agus tugadh na treoracha úsáide ábhartha dóibh chun an córas intleachta saorga a úsáid dá dtagraítear in Airteagal 13; tabharfaidh an soláthraí nó an soláthraí ionchasach agus an t-úsáideoir gairmiúil nó úsáideoir gairmiúil ionchasach comhaontú i gcrích ina sonrófar a róil agus a bhfreagrachtaí d’fhonn a áirithiú go gcomhlíonfar na forálacha maidir le tástáil i bhfíordhálaí faoin Rialachán seo agus faoi dhlí infheidhme eile de chuid an Aontais agus de chuid na mBallstát;
(i)
tá toiliú feasach tugtha ag na daoine is ábhar don tástáil i bhfíordhálaí i gcomhréir le hAirteagal 61, nó i gcás fhorfheidhmiú an dlí, i gcás ina gcuirfeadh lorg toiliú feasach cosc ar thástáil an chórais intleachta saorga, ní bheidh aon éifeacht dhiúltach ag an tástáil féin ná ag toradh na tástála sna fíordhálaí ar na daoine is ábhar, agus scriosfar a sonraí pearsanta tar éis don tástáil a bheith déanta;
(j)
déanann an soláthraí nó an soláthraí ionchasach, mar aon le húsáideoirí gairmiúla nó úsáideoirí gairmiúla ionchasacha, maoirseacht éifeachtach ar an tástáil i bhfíordhálaí trí dhaoine atá cáilithe go cuí sa réimse ábhartha agus a bhfuil an acmhainneacht, an oiliúint agus an t-údarás is gá acu chun a gcúraimí a dhéanamh;
(k)
an méid a dhéanann an córas intleachta saorga a thuar, a mholadh nó a chinneadh, is féir é sin a aisiompú go héifeachtach agus neamhaird a thabhairt air.
|
|
Airteagal 60 5.
|
Féadfaidh aon duine is ábhar don tástáil i bhfíordhálaí, nó a n-ionadaí arna ainmniú de réir dlí, de réir mar is iomchuí, gan dochar dó dá dheasca agus gan aon dualgas a bheith orthu bonn cirt a thabhairt, tarraingt siar ón tástáil tráth ar bith trína dtoiliú feasach a chúlghairm agus féadfaidh siad a iarraidh go ndéanfar a sonraí pearsanta a scriosadh láithreach agus go buan. Ní dhéanfaidh tarraingt siar an toilithe fheasaigh difear do na ghníomhaíochtaí a rinneadh cheana.
|
|
Airteagal 60 6.
|
I gcomhréir le hAirteagal 75, tabharfaidh na Ballstáit dá n-údaráis faireachais margaidh na cumhachtaí chun a cheangal ar sholáthraithe agus ar sholáthraithe ionchasacha faisnéis a sholáthar, cianchigireachtaí gan réamhfhógra nó cigireachtaí ar an láthair a dhéanamh, agus seiceálacha a dhéanamh ar dhéanamh na tástála i bhfíordhálaí agus ar na córais intleachta saorga ardriosca ghaolmhara. Úsáidfidh na húdaráis faireachais margaidh na cumhachtaí sin chun a áirithiú go ndéanfar tástáil a fhorbairt go sábháilte i bhfíordhálaí.
|
|
Airteagal 60 7.
|
Aon teagmhas tromchúiseach a shainaithnítear le linn na tástála i bhfíordhálaí, déanfar é a thuairisciú don údarás náisiúnta faireachais margaidh i gcomhréir le hAirteagal 73. Glacfaidh an soláthraí nó an soláthraí ionchasach bearta maolaithe láithreacha nó, ina éagmais sin, cuirfidh sé an tástáil i bhfíordhálaí ar fionraí go dtí go ndéanfar an maolú sin nó cuirfidh sé deireadh léi. Bunóidh an soláthraí nó an soláthraí ionchasach nós imeachta chun an córas intleachta saorga a aisghairm go pras i gcás ina ndéanfaí an tástáil i bhfíordhálaí a fhoirceannadh.
|
|
Airteagal 60 8.
|
Má dhéantar an tástáil i bhfíordhálaí a fhoirceannadh nó a fhionraí, tabharfaidh soláthraithe nó soláthraithe ionchasacha fógra don údarás náisiúnta faireachais margaidh sa Bhallstát nó sna Ballstáit ina bhfuil an tástáil le déanamh faoi sin agus faoi na torthaí deiridh.
|
|
Airteagal 60 9.
|
Beidh an soláthraí nó an soláthraí ionchasach faoi dhliteanas faoin dlí is infheidhme de chuid an Aontais agus na mBallstát maidir le dliteanais i leith aon damáiste a dhéantar le linn a rannpháirtíochta sa tástáil i bhfíordhálaí.
|
,
1.1 Tagairtí
1.2 Ceanglais aitheanta
1.3 Caighdeáin ghaolmhara
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) (SCF)
- AI TEVV Reporting (SCF)
- AI TEVV Reporting (SCF)
- Serious Incident Reporting For AI & Autonomous Technologies (SCF)
2. Ceanglais aitheanta
Ceanglais
| Foinse |
Ceanglas |
3. Caighdeáin ghaolmhara
Caighdeáin
| Foinse |
Ceanglas |
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV)
Description
Mechanisms exist to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable Artificial Intelligence (AI) and Autonomous Technologies (AAT)-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Information Assurance (IA) Program ∙ AI TEVV checklist for AI tools (test accuracy, validate outputs, verify security) ∙ AI governance program
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Information Assurance (IA) Program ∙ Formal AI TEVV process for AI systems ∙ AI governance program
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Information Assurance (IA) Program ∙ Formal AI TEVV framework aligned to NIST AI RMF Measure function ∙ AI testing tools (e.g., IBM OpenScale, Great Expectations) ∙ AI governance program
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program ∙ NIST AI RMF Measure function ∙ Third-party AI testing for high-risk systems ∙ AI testing integrated with CI/CD pipelines
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Information Assurance (IA) Program ∙ Enterprise AI TEVV program (NIST AI RMF Measure function) ∙ Independent AI testing and evaluation for high-risk systems ∙ AI TEVV integrated with MLOps and CI/CD ∙ EU AI Act conformity assessment (if applicable)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to implement Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) practices to enable AAT-related security, resilience and compliance-related conformity testing throughout the lifecycle of the AAT.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
AI TEVV Reporting
Description
Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report AI testing results to management
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Document and communicate AI test results to stakeholders
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI TEVV reporting process ∙ Standardized reporting templates
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI TEVV reporting program ∙ Regular stakeholder briefings ∙ Findings tracking
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI TEVV reporting framework ∙ Automated test result dashboards ∙ Board-level AI risk reporting
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
AI TEVV Reporting
Description
Mechanisms exist to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report AI testing results to management
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Document and communicate AI test results to stakeholders
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI TEVV reporting process ∙ Standardized reporting templates
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI TEVV reporting program ∙ Regular stakeholder briefings ∙ Findings tracking
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI TEVV reporting framework ∙ Automated test result dashboards ∙ Board-level AI risk reporting
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control: ▪ A reasonable person would conclude this control requires a structured process. ▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
SCR-CMM Level 2 criteria definitions are not available for this control: ▪ A reasonable person would conclude a well-defined and standardized process is required. ▪ At this level of maturity, the “requirements-driven” nature of performing the control is focused on a localized and/or regionalized implementation, not uniform and consistent across the organization. ▪ Requirements are narrowly scoped for applicability and are primarily derived from compliance obligations (e.g., laws, regulations and contracts).
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report the status and results of Artificial Intelligence Test, Evaluation, Validation & Verification (AI TEVV) to relevant stakeholders, including governing bodies, as required.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|
SCF
|
Serious Incident Reporting For AI & Autonomous Technologies
Description
Mechanisms exist to report any serious incident involving operational Artificial Intelligence (AI) and Autonomous Technologies (AAT) to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report serious AI incidents to relevant authorities as required
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Incident reporting procedure for serious AI incidents
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal serious AI incident reporting process ∙ Regulatory reporting contacts
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI incident response plan with regulatory reporting requirements ∙ Legal team involvement
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI incident response program ∙ Automated regulatory notification workflows ∙ Legal and compliance integration
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include: ▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation. ▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control and risk assessments). ▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners. ▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD). ▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards and procedures & guidelines). ▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist: ▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function. ▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners. ▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities. ▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform). ▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls). ▪ An implemented and operational capability exists to report any serious incident involving operational AAT to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions: ▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control. ▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions: ▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control. ▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies. ▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
|
|