+Serious Incident Reporting For AI & Autonomous Technologies
|
Serious Incident Reporting For AI & Autonomous Technologies
Description
Mechanisms exist to report any serious incident involving operational Artificial Intelligence (AI) and Autonomous Technologies (AAT) to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Report serious AI incidents to relevant authorities as required
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ Incident reporting procedure for serious AI incidents
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal serious AI incident reporting process
∙ Regulatory reporting contacts
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI incident response plan with regulatory reporting requirements
∙ Legal team involvement
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI incident response program
∙ Automated regulatory notification workflows
∙ Legal and compliance integration
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
Artificial Intelligence and Autonomous Technology (AAT) domain capabilities are ad hoc and inconsistent. Capability criteria associated with this control may include:
▪ Policies, standards & procedures associated with AAT domain capabilities provide limited coverage due to the depth and breadth of the existing documentation.
▪ AAT-related processes are expected to follow the organization's existing processes (e.g., incident response, asset management, change control, risk assessments, etc.).
▪ No formal Governance, Risk & Compliance (GRC) team exists to provide AAT oversight, where the Chief Information Officer (CIO), or similar function, governs technology decisions what is acceptable for AAT within the organization.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to report any serious incident involving operational AAT to relevant authorities as to when and where the serious incident occurred, in accordance with mandated reporting timelines.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
1. Overview
1.1 References
1.2 Identified Requirements
1.3 Related Regulations
2. Identified Requirements
Requirements
| Source |
Requirement |
3. Related Regulations
Regulations
| Source |
Regulation |
|
EULAW
|
Article 55 Obligations of providers of general-purpose AI models with systemic risk
Article 55
Obligations of providers of general-purpose AI models with systemic risk
1. In addition to the obligations listed in Articles 53 and 54, providers of general-purpose AI models with systemic risk shall:
|
(a)
|
perform model evaluation in accordance with standardised protocols and tools reflecting the state of the art, including conducting and documenting adversarial testing of the model with a view to identifying and mitigating systemic risks;
|
|
(b)
|
assess and mitigate possible systemic risks at Union level, including their sources, that may stem from the development, the placing on the market, or the use of general-purpose AI models with systemic risk;
|
|
(c)
|
keep track of, document, and report, without undue delay, to the AI Office and, as appropriate, to national competent authorities, relevant information about serious incidents and possible corrective measures to address them;
|
|
(d)
|
ensure an adequate level of cybersecurity protection for the general-purpose AI model with systemic risk and the physical infrastructure of the model.
|
2. Providers of general-purpose AI models with systemic risk may rely on codes of practice within the meaning of Article 56 to demonstrate compliance with the obligations set out in paragraph 1 of this Article, until a harmonised standard is published. Compliance with European harmonised standards grants providers the presumption of conformity to the extent that those standards cover those obligations. Providers of general-purpose AI models with systemic risks who do not adhere to an approved code of practice or do not comply with a European harmonised standard shall demonstrate alternative adequate means of compliance for assessment by the Commission.
3. Any information or documentation obtained pursuant to this Article, including trade secrets, shall be treated in accordance with the confidentiality obligations set out in Article 78.
|
|
EULAW
|
Article 60 Testing of high-risk AI systems in real world conditions outside AI regulatory sandboxes
Article 60
Testing of high-risk AI systems in real world conditions outside AI regulatory sandboxes
1. Testing of high-risk AI systems in real world conditions outside AI regulatory sandboxes may be conducted by providers or prospective providers of high-risk AI systems listed in Annex III, in accordance with this Article and the real-world testing plan referred to in this Article, without prejudice to the prohibitions under Article 5.
The Commission shall, by means of implementing acts, specify the detailed elements of the real-world testing plan. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 98(2).
This paragraph shall be without prejudice to Union or national law on the testing in real world conditions of high-risk AI systems related to products covered by Union harmonisation legislation listed in Annex I.
2. Providers or prospective providers may conduct testing of high-risk AI systems referred to in Annex III in real world conditions at any time before the placing on the market or the putting into service of the AI system on their own or in partnership with one or more deployers or prospective deployers.
3. The testing of high-risk AI systems in real world conditions under this Article shall be without prejudice to any ethical review that is required by Union or national law.
4. Providers or prospective providers may conduct the testing in real world conditions only where all of the following conditions are met:
|
(a)
|
the provider or prospective provider has drawn up a real-world testing plan and submitted it to the market surveillance authority in the Member State where the testing in real world conditions is to be conducted;
|
|
(b)
|
the market surveillance authority in the Member State where the testing in real world conditions is to be conducted has approved the testing in real world conditions and the real-world testing plan; where the market surveillance authority has not provided an answer within 30 days, the testing in real world conditions and the real-world testing plan shall be understood to have been approved; where national law does not provide for a tacit approval, the testing in real world conditions shall remain subject to an authorisation;
|
|
(c)
|
the provider or prospective provider, with the exception of providers or prospective providers of high-risk AI systems referred to in points 1, 6 and 7 of Annex III in the areas of law enforcement, migration, asylum and border control management, and high-risk AI systems referred to in point 2 of Annex III has registered the testing in real world conditions in accordance with Article 71(4) with a Union-wide unique single identification number and with the information specified in Annex IX; the provider or prospective provider of high-risk AI systems referred to in points 1, 6 and 7 of Annex III in the areas of law enforcement, migration, asylum and border control management, has registered the testing in real-world conditions in the secure non-public section of the EU database according to Article 49(4), point (d), with a Union-wide unique single identification number and with the information specified therein; the provider or prospective provider of high-risk AI systems referred to in point 2 of Annex III has registered the testing in real-world conditions in accordance with Article 49(5);
|
|
(d)
|
the provider or prospective provider conducting the testing in real world conditions is established in the Union or has appointed a legal representative who is established in the Union;
|
|
(e)
|
data collected and processed for the purpose of the testing in real world conditions shall be transferred to third countries only provided that appropriate and applicable safeguards under Union law are implemented;
|
|
(f)
|
the testing in real world conditions does not last longer than necessary to achieve its objectives and in any case not longer than six months, which may be extended for an additional period of six months, subject to prior notification by the provider or prospective provider to the market surveillance authority, accompanied by an explanation of the need for such an extension;
|
|
(g)
|
the subjects of the testing in real world conditions who are persons belonging to vulnerable groups due to their age or disability, are appropriately protected;
|
|
(h)
|
where a provider or prospective provider organises the testing in real world conditions in cooperation with one or more deployers or prospective deployers, the latter have been informed of all aspects of the testing that are relevant to their decision to participate, and given the relevant instructions for use of the AI system referred to in Article 13; the provider or prospective provider and the deployer or prospective deployer shall conclude an agreement specifying their roles and responsibilities with a view to ensuring compliance with the provisions for testing in real world conditions under this Regulation and under other applicable Union and national law;
|
|
(i)
|
the subjects of the testing in real world conditions have given informed consent in accordance with Article 61, or in the case of law enforcement, where the seeking of informed consent would prevent the AI system from being tested, the testing itself and the outcome of the testing in the real world conditions shall not have any negative effect on the subjects, and their personal data shall be deleted after the test is performed;
|
|
(j)
|
the testing in real world conditions is effectively overseen by the provider or prospective provider, as well as by deployers or prospective deployers through persons who are suitably qualified in the relevant field and have the necessary capacity, training and authority to perform their tasks;
|
|
(k)
|
the predictions, recommendations or decisions of the AI system can be effectively reversed and disregarded.
|
5. Any subjects of the testing in real world conditions, or their legally designated representative, as appropriate, may, without any resulting detriment and without having to provide any justification, withdraw from the testing at any time by revoking their informed consent and may request the immediate and permanent deletion of their personal data. The withdrawal of the informed consent shall not affect the activities already carried out.
6. In accordance with Article 75, Member States shall confer on their market surveillance authorities the powers of requiring providers and prospective providers to provide information, of carrying out unannounced remote or on-site inspections, and of performing checks on the conduct of the testing in real world conditions and the related high-risk AI systems. Market surveillance authorities shall use those powers to ensure the safe development of testing in real world conditions.
7. Any serious incident identified in the course of the testing in real world conditions shall be reported to the national market surveillance authority in accordance with Article 73. The provider or prospective provider shall adopt immediate mitigation measures or, failing that, shall suspend the testing in real world conditions until such mitigation takes place, or otherwise terminate it. The provider or prospective provider shall establish a procedure for the prompt recall of the AI system upon such termination of the testing in real world conditions.
8. Providers or prospective providers shall notify the national market surveillance authority in the Member State where the testing in real world conditions is to be conducted of the suspension or termination of the testing in real world conditions and of the final outcomes.
9. The provider or prospective provider shall be liable under applicable Union and national liability law for any damage caused in the course of their testing in real world conditions.
|
|
EULAW
|
Article 73 Reporting of serious incidents
Article 73
Reporting of serious incidents
1. Providers of high-risk AI systems placed on the Union market shall report any serious incident to the market surveillance authorities of the Member States where that incident occurred.
2. The report referred to in paragraph 1 shall be made immediately after the provider has established a causal link between the AI system and the serious incident or the reasonable likelihood of such a link, and, in any event, not later than 15 days after the provider or, where applicable, the deployer, becomes aware of the serious incident.
The period for the reporting referred to in the first subparagraph shall take account of the severity of the serious incident.
3. Notwithstanding paragraph 2 of this Article, in the event of a widespread infringement or a serious incident as defined in Article 3, point (49)(b), the report referred to in paragraph 1 of this Article shall be provided immediately, and not later than two days after the provider or, where applicable, the deployer becomes aware of that incident.
4. Notwithstanding paragraph 2, in the event of the death of a person, the report shall be provided immediately after the provider or the deployer has established, or as soon as it suspects, a causal relationship between the high-risk AI system and the serious incident, but not later than 10 days after the date on which the provider or, where applicable, the deployer becomes aware of the serious incident.
5. Where necessary to ensure timely reporting, the provider or, where applicable, the deployer, may submit an initial report that is incomplete, followed by a complete report.
6. Following the reporting of a serious incident pursuant to paragraph 1, the provider shall, without delay, perform the necessary investigations in relation to the serious incident and the AI system concerned. This shall include a risk assessment of the incident, and corrective action.
The provider shall cooperate with the competent authorities, and where relevant with the notified body concerned, during the investigations referred to in the first subparagraph, and shall not perform any investigation which involves altering the AI system concerned in a way which may affect any subsequent evaluation of the causes of the incident, prior to informing the competent authorities of such action.
7. Upon receiving a notification related to a serious incident referred to in Article 3, point (49)(c), the relevant market surveillance authority shall inform the national public authorities or bodies referred to in Article 77(1). The Commission shall develop dedicated guidance to facilitate compliance with the obligations set out in paragraph 1 of this Article. That guidance shall be issued by 2 August 2025, and shall be assessed regularly.
8. The market surveillance authority shall take appropriate measures, as provided for in Article 19 of Regulation (EU) 2019/1020, within seven days from the date it received the notification referred to in paragraph 1 of this Article, and shall follow the notification procedures as provided in that Regulation.
9. For high-risk AI systems referred to in Annex III that are placed on the market or put into service by providers that are subject to Union legislative instruments laying down reporting obligations equivalent to those set out in this Regulation, the notification of serious incidents shall be limited to those referred to in Article 3, point (49)(c).
10. For high-risk AI systems which are safety components of devices, or are themselves devices, covered by Regulations (EU) 2017/745 and (EU) 2017/746, the notification of serious incidents shall be limited to those referred to in Article 3, point (49)(c) of this Regulation, and shall be made to the national competent authority chosen for that purpose by the Member States where the incident occurred.
11. National competent authorities shall immediately notify the Commission of any serious incident, whether or not they have taken action on it, in accordance with Article 20 of Regulation (EU) 2019/1020.
|
Linked Issues
- Secure Controls Framework -
"The SCF is the Common Controls Framework™ (CCF), the world's most comprehensive cybersecurity and data privacy metaframework - it is also free to use. The entire concept is building secure, compliant and resilient capabilities in the most efficient and cost-effective manner possible.
The SCF is more than just a unified control catalog, since its included content creates a playbook for Governance, Risk & Compliance (GRC) capabilities. Used globally by organizations of every size, the SCF is a robust and scalable solution for security, compliance and resilience controls. As a comprehensive security framework, the SCF maps 1,400+ controls across 200+ laws, regulations, and industry frameworks so you can implement once and comply everywhere.
Like it or not, cybersecurity is a protracted war on an asymmetric battlefield, where the threats are everywhere and as defenders we have to make the effort to work together to help improve cybersecurity and data privacy practices, since we all suffer when massive data breaches occur or when cyber attacks have physical impacts. Hackers share information on attack methods with other hackers, so why shouldn’t the good guys share information on how to best protect an organization? We decided to take action and make a difference, since we feel it is too important to wait for someone else to fix the problems that exist.
The SCF is made up of volunteers, mainly specialists within the cybersecurity profession, who focus on GRC and the cybersecurity side of data privacy. These are auditors, engineers, architects, incident responders, consultants and other specialists who live and breathe these topics on a daily basis. The end product is "expert-derived content" that makes up the SCF." https://securecontrolsframework.com/
Terms & Conditions
The SCF End User License Agreement (EULA) governs the use of the Secure Controls Framework® (SCF) under the Creative Commons Attribution-No Derivatives 4.0 International Public License.
|