OPS.1.1.6 Software Tests and Approvals

Before software tests are carried out, corresponding framework conditions MUST be specified within an organisation in accordance with the protection needs, organisational units, technical options, and test environments at hand. Software MUST be tested on the basis of a software requirements catalogue. If there is also a requirements specification, it MUST be taken into account as well.

Test cases MUST be selected in such a way that they test all the software functions at hand in the most representative way possible. In addition, negative tests SHOULD be included to ensure software does not contain any unwanted functions.

The test environment MUST be selected to be as representative as possible of all the device models and operating system environments used in the organisation in question. Testing SHOULD also determine whether software is compatible with and functional on the operating systems used in their present configurations.

Functional software tests MUST verify the proper and complete functioning of software. Functional software tests MUST be performed in a manner that will not impact production operations.

The results of software tests MUST be assessed. A gap analysis SHOULD be performed using the defined specifications. Such assessments MUST be documented.

The technical organisational unit responsible MUST approve software as soon as it has passed corresponding tests. The approval MUST be documented by means of an approval confirmation.

The approving organisational unit MUST verify whether the software has been tested in accordance with the relevant requirements. The results of the software tests MUST match the previously specified expectations. It MUST also be verified that the legal and organisational requirements at hand have been met.

Software tests MUST be performed to verify that all essential non-functional requirements are met. In particular, security-specific software tests MUST be performed if the application in question includes security-critical functions. The test cases carried out MUST be documented along with their results.

If production data containing sensitive information is used for software tests, it MUST be appropriately protected during the testing. If this data contains personal information, it must be anonymised at minimum. Personal test data SHOULD be completely anonymised if possible. If references to persons could be derived from the test data, the Data Protection Officer and (if applicable) the Employee Representatives MUST be consulted.

The Process Owner SHOULD inform software testers of the test types to be performed and the software areas to be tested. Furthermore, the software testers SHOULD be informed of the use cases and possible further requirements of the software.

Particular criteria SHOULD be considered when selecting software testers. Software testers SHOULD have the required professional qualifications.

If custom software is to be reviewed at the source code level, the testers SHOULD have sufficient expertise in the programming language and development environment to be tested. The source code SHOULD NOT be reviewed exclusively by testers who were also involved in the creation of the source code.

An acceptance plan SHOULD document the test types to be performed, the test cases considered, and the expected results. Furthermore, the acceptance plan SHOULD include the corresponding approval criteria. A procedure SHOULD be defined for the possibility of approval being denied.

Regression testing SHOULD be performed when software has been changed. It SHOULD check whether previously existing security mechanisms and settings have been unintentionally changed by the update. Regression tests SHOULD be performed in full and include extensions and auxiliary resources. If test cases are omitted, this SHOULD be justified and documented. The test cases carried out SHOULD be documented along with their results.

Software SHOULD only be tested in a test environment intended for this purpose. The test environment SHOULD be operated separately from the production environment. The architectures and mechanisms used in the test environment SHOULD be documented. The documentation SHOULD also specify what is to be done with the test environment once the software tests are complete.

Software installations SHOULD be checked according to the rules for the installation and configuration of software (see module APP.6 General Software). If available, the installation and configuration documentation SHOULD also be checked.

Penetration tests SHOULD be performed on applications and/or IT systems with increased protection needs. A concept for penetration tests SHOULD be created. In addition to the deployed test methods, the concept for penetration tests SHOULD document corresponding success criteria.

Penetration tests SHOULD be performed in accordance with the framework conditions of the penetration test concept. The vulnerabilities detected during penetration tests SHOULD be classified and documented.

If testers require access to particularly sensitive information, the organisation in question SHOULD obtain evidence of their integrity and reputation. When dealing with classified material, software testers SHOULD be subjected to security vetting in line with the German Security Screening Act (SÜG). The CISO SHOULD involve the organisation's Confidentiality Officer or Security Representative in this regard.