+3.5 Continuously Evaluate and Improve
---+3.5.1 Evaluation Inputs
---+3.5.2 Performance Metrics
---+3.5.3 Measurement in Support of Customer Experience Outcomes
|
3.5 Continuously Evaluate and Improve
Continuous improvement is a critical tool for keeping pace with the threat and technology environment and identifying programmatic gaps that need to be addressed to balance risk management objectives. For instance, an organization may determine that a portion of the target population intended to be served by the online service does not have access to affordable high-speed internet services, which are needed to support remote identity proofing. The organization could bridge this gap by establishing a program that offers local, in-person proofing services within the community. This could involve providing appointments with proofing agents who can meet individuals at more accessible locations, such as their local community center, the nearest post office, a partner business facility, or even at the individual’s home. To address the shifting environment in which they operate and more rapidly address service capability gaps, organizations SHALL implement a continuous evaluation and improvement program that leverages input from end users who have interacted with the identity management system as well as performance metrics for the online service. This program SHALL be documented, including the metrics that are collected, the sources of data required to enable performance evaluation, and the processes in place for taking timely actions based on the continuous improvement process. This program and its effectiveness SHOULD be assessed on a regular basis to ensure that outcomes are being achieved and that programs are addressing issues in a timely manner. Additionally, organizations SHALL monitor the evolving threat landscape to stay informed of the latest threats and fraud tactics. Organizations SHALL regularly assess the effectiveness of current security measures and fraud detection capabilities against the latest threats and fraud tactics.
1. Přehled
| Název |
Standard |
|
3.5.1 Evaluation Inputs
|
To fully understand the performance of their identity system, organizations will need to identify critical inputs to their continuous evaluation process. At a minimum, these inputs SHALL include:
Integrated CSP, IdP, and authentication functions as well as validation, verification, and fraud management systems, as appropriate
Customer feedback mechanisms, such as complaint processes, helpdesk statistics, and other user feedback (e.g., surveys, interviews, or focus groups)
Threat analysis, threat reporting, and threat intelligence feeds that are available
Fraud trends, fraud investigation results, and fraud metrics as available
The results of ongoing customer experience assessments and privacy assessments
RPs SHALL document their metrics, reporting requirements, and data inputs for any CSP, IdP, or other integrated identity service to ensure that expectations are appropriately communicated to partners and vendors.
|
|
3.5.2 Performance Metrics
|
The exact metrics available to organizations will vary based on the technologies, architectures, and deployment methods they use. Additionally, the availability and usefulness of certain metrics will vary over time. Therefore, these guidelines do not attempt to define a comprehensive set of metrics for all scenarios. Table 4 provides a set of recommended metrics that organizations SHOULD track as part of their continuous evaluation program. However, organizations are not constrained by this table and SHOULD implement metrics based on their specific systems, technology, and program needs. See [SP800-55V2] for more information on identifying additional performance metrics. In Table 4, all references to unique users include both legitimate users and imposters. Table 4. Performance metrics
| Title |
Description |
Type |
| Pass Rate (Overall) |
Percentage of unique users who successfully complete identity proofing |
Proofing |
| Pass Rate (Per Proofing Type) |
Percentage of unique users who are successfully proofed for each offered type (i.e., remote unattended, remote attended, on-site attended, on-site unattended) |
Proofing |
| Fail Rate (Overall) |
Percentage of unique users who start the identity proofing process but are unable to successfully complete all of the steps |
Proofing |
| Estimated Adjusted Fail Rate |
Percentage of failures adjusted to account for identity proofing attempts that are suspected to be fraudulent |
Proofing |
| Fail Rate (Per Proofing Type) |
Percentage of unique users who do not complete proofing due to a process failure for each offered type (i.e., remote unattended, remote attended, on-site attended, on-site unattended) |
Proofing |
| Abandonment Rate (Overall) |
Percentage of unique users who start the identity proofing process but do not complete it without failing a process |
Proofing |
| Abandonment Rate (Per Proofing Type) |
Percentage of unique users who start a specific type of identity proofing process but do not complete it without failing a process |
Proofing |
| Failure Rates (Per Proofing Process Step) |
Percentage of unique users who are unsuccessful at completing each identity proofing step in a CSP process |
Proofing |
| Completion Times (Per Proofing Type) |
Average time that it takes a user to complete each defined proofing type offered as part of an identity service |
Proofing |
| Authenticator Type Usage |
Percentage of subscribers who have an active authenticator by each type available |
Authentication |
| Authentication Failures |
Percentage of authentication events that fail (not to include authentication attempts that are successful after re-entry of an authenticator output) |
Authentication |
| Account Recovery Attempts |
The number of account or authenticator recovery processes initiated by subscribers |
Authentication |
| Confirmed Unauthorized Access or Fraud |
Percentage of total transaction events (i.e., identity proofing + authentication events) that the organization determines to be unauthorized or fraudulent through analysis or self-reporting |
Fraud |
| Suspected Unauthorized Access or Fraud |
Percentage of total transaction events (i.e., identity proofing + authentication events) that are suspected to be unauthorized or fraudulent |
Fraud |
| Reported Unauthorized Access or Fraud |
Percentage of total transaction events (i.e., identity proofing + authentication events) reported to be unauthorized or fraudulent by users |
Fraud |
| Unauthorized Access or Fraud (Per Proofing Type) |
Number of identity proofing events that are suspected or reported as being fraudulent for each available type of proofing |
Fraud |
| Unauthorized Access or Fraud (Per Authentication Type) |
Number of authentication events that are suspected or reported to be unauthorized or fraudulent by each available type of authentication |
Fraud |
| Help Desk Calls |
Number of calls received by the CSP or identity service |
Customer Experience |
| Help Desk Calls (Per Type) |
Number of calls received related to each offered service (e.g., proofing failures, authenticator resets, complaints) |
Customer Experience |
| Help Desk Resolution Times |
Average length of time that it takes to resolve a complaint or help desk ticket |
Customer Experience |
| Customer Satisfaction Surveys |
The results of customer feedback surveys conducted by CSPs, RP, or both |
Customer Experience |
| Redress Requests |
The number of redress requests received related to the identity management system |
Customer Experience |
| Redress Resolution Times |
The average time it takes to resolve redress requests related to the identity management system |
Customer Experience |
The data used to generate continuous evaluation metrics may not always reside with the identity program or the organizational entity responsible for identity management systems. The intent of these metrics is to integrate with existing data sources whenever possible to collect information that is critical to identity program evaluation. For example, customer service representative (CSR) teams may already have substantial information on customer requests, complaints, or concerns. Organizations that implement and maintain identity management systems are expected to coordinate with these teams to acquire the information needed to discern identity management system-related complaints or issues.
|
|
3.5.3 Measurement in Support of Customer Experience Outcomes
|
A primary goal of continuous improvement is to enhance customer experience, usability, and accessibility outcomes for different user populations. As a result, the metrics collected by organizations SHOULD be further evaluated to provide insights into the performance of their identity management systems for their supported communities. Where possible, these efforts SHOULD avoid the collection of additional personal information and instead use informed analyses of proxy data to identify potential performance issues. This can include comparing and filtering the metrics to understand deviations in performance across different user populations based on other available data, such as zip code, geographic region, age, or sex.
|
1.1 Odkazy
1.2 Identifikované požadavky
1.3 Související regulace
2. Identifikované požadavky
Požadavky
| Zdroj |
Požadavek |
3. Související regulace
- NIST -
EN: Publications of the National Institute of Standards and Technology (NIST), U.S. Department of Commerce, https://www.nist.gov/, from their electronic versions (OSCAL; SP 800-63-4 from its HTML version, https://pages.nist.gov/800-63-4/). Works of NIST employees are not subject to copyright protection in the United States (17 U.S.C. § 105); the OSCAL content is dedicated to the public domain worldwide (CC0 1.0); outside the United States NIST grants the right to reprint its technical series publications in all formats (NIST copyright statement). Republished courtesy of the National Institute of Standards and Technology; each document names its citation. Divided into sections and converted into the GRCA format, the wording unchanged. Only the versions published by NIST are authoritative.
DE: Veröffentlichungen des National Institute of Standards and Technology (NIST), U.S. Department of Commerce, https://www.nist.gov/, aus ihren elektronischen Fassungen (OSCAL; SP 800-63-4 aus ihrer HTML-Fassung, https://pages.nist.gov/800-63-4/). Werke von NIST-Beschäftigten sind in den USA urheberrechtlich nicht geschützt (17 U.S.C. § 105); die OSCAL-Inhalte sind weltweit gemeinfrei gestellt (CC0 1.0); außerhalb der USA räumt das NIST das Recht ein, seine Technical-Series-Publikationen in allen Formaten neu zu veröffentlichen (Copyright-Hinweis des NIST). „Republished courtesy of the National Institute of Standards and Technology“; jedes Dokument nennt seine Zitierangabe. In Abschnitte aufgeteilt und in das GRCA-Format umgewandelt, der Wortlaut unverändert. Maßgeblich sind allein die vom NIST veröffentlichten Fassungen.
|