+I-DM-B Stream B: Metrics and Feedback
---+I-DM-1-B Define basic defect metrics
---+I-DM-2-B Define advanced defect metrics
---+I-DM-3-B Use metrics to improve the security strategy

I-DM-B Stream B: Metrics and Feedback

Defect tracking can drive the improvement of security activities within the organization through metrics and feedback.

1. Prehľad

I-DM-1-B Define basic defect metrics

Maturity level 1 objective of the practice: All defects are tracked within each project.

Benefit: Identification of quick wins derived from available defect information

Short description: Regularly go over previously recorded security defects and derive quick wins from basic metrics.

Once per defined period of time (typically at least once per year), go over your both resolved and still open recorded security defects in every team and extract basic metrics from the available data. These might include:

  • The total number of defects versus total number of verification activities. This could give you an idea whether you're looking for defects with an adequate intensity and quality.
  • The software components the defects reside in. This is indicative of where attention might be most required, and where security flaws might be more likely to appear in the future again.
  • The type or category of the defect, which suggests areas where the development team needs further training.
  • The severity of the defect, which can help the team understand the software's risk exposure.

Identify and carry out sensible quick win activities which you can derive from the newly acquired knowledge. These might include things like a knowledge sharing session about one particular vulnerability type or carrying out or automating a security scan.

Results

  • result1
  • result2

Metrics

  • metric1
  • metric2

Costs

TBS.

Personnel

  • Developer
  • Security Champion
I-DM-2-B Define advanced defect metrics

Maturity level 2 objective of the practice: Defect tracking is used to influence the deployment process.

Benefit: Improved learning from security defects in your organization

Short description: Collect standardized defect management metrics and use these also for prioritization of centrally driven initiatives.

Define, collect and calculate unified metrics across the whole organization. These might include:

  • Total amount of verification activities and identified defects.
  • Types and severities of identified defects.
  • Time to detect and time to resolve defects.
  • Windows of exposure of defects being present on live systems.
  • Number of regressions / reopened vulnerabilities.
  • Coverage of verification activities for particular software components.
  • Amount of accepted risk.
  • Ratio of security incidents caused due to unknown or undocumented security defects.

Generate a regular (e.g. monthly) report for a suitable audience. This would typically reach audiences like managers, security officers, and engineers. Use the information in the report as an input for your security strategy, e.g. improving trainings or security verification activities.

Share the most prominent or interesting technical details about security defects including the fixing strategy to other teams once these defects are fixed, e.g. in a regular knowledge sharing meeting. This will help scale the learning effect from defects to the whole organization and limit their occurrence in the future.

Results

  • result1
  • result2

Metrics

  • metric1
  • metric2

Costs

TBS.

Personnel

  • Security Officer
  • Security Champion
  • Manager
I-DM-3-B Use metrics to improve the security strategy

Maturity level 3 objective of the practice: Defect tracking across multiple components is used to help reduce the number of new defects.

Benefit: Optimized security strategy based on defect information

Short description: Continuously improve your security defect management metrics and correlate it with other sources.

Regularly (at least once per year) revisit the defect management metrics you're collecting and compare the effort needed to collect and track these to the expected outcomes. Make knowledgeable decisions about removing metrics which don't deliver the overall expected value. Wherever possible, include and automate verification activities for the quality of the collected data and ensure sustainable improvement if any differences are detected.

Aggregate the data with your threat intelligence and incident management metrics and use the results as input for other initiatives over the whole organization, such as:

  • Planning security trainings for various personnel
  • Improvement of security verification activities for both internally and externally developed collected
  • Supply chain management, e.g. carrying out security audits of partner organizations
  • Monitoring of attacks against your infrastructure and applications
  • Investing in security infrastructure or compensating controls
  • Staffing your security team and setting up the security budget

Results

  • result1
  • result2

Metrics

  • metric1
  • metric2

Costs

TBS.

Personnel

  • Security Officer
  • Security Engineer
  • Manager

1.1 Odkazy

1.2 Identifikované požiadavky

1.3 Súvisiace predpisy

2. Identifikované požiadavky

Požiadavky

3. Súvisiace predpisy

Predpisy
  • OWASP -

    EN: Publications of the OWASP Foundation, https://owasp.org/. OWASP® is a registered trademark of the OWASP Foundation, Inc.; GRCA is not affiliated with, nor endorsed by, the OWASP Foundation. Only the versions published by OWASP are authoritative.

    DE: Veröffentlichungen der OWASP Foundation, https://owasp.org/. OWASP® ist eine eingetragene Marke der OWASP Foundation, Inc.; GRCA steht in keiner Verbindung zur OWASP Foundation und wird von ihr nicht unterstützt. Maßgeblich sind allein die von OWASP veröffentlichten Fassungen.