Capability Maturity Models

Capability development is not a point in time activity. It must be sustained and improved throughout the life of an organisation. This requires mechanisms for continually monitoring and measuring the capability maturity of the organisation. A mature governance framework enables the leadership to set objectives, monitor performance, match performance with internal and external drivers and constraints to derive future projections.

A capability maturity model typically comprises of a set of characteristics, attributes, indicators, or patterns that represent capability and progression in a particular discipline. Principally, the models must be able to provide insights to some of the concerns of CSEs and national bodies, as listed below:

  • What are the maturity levels of CSEs in comparison with established benchmarks in governance, risk management, enterprise architectures, system lifecycle management, IT and infosec capabilities, technology implementations, operations and support processes, workforce characteristics?

  • What must be improved?

  • What needs to be measured and assessed to enable improvement?

  • How comprehensive are the enterprise policies, practices and processes?

  • How good is the visibility, oversight, and control across the enterprise on the compliance to policies?

In general, CMMs have the following characteristics:

  • collect the best practices.

  • are developed by a collaboration of experts from diverse backgrounds.

  • consider the dispersion in size, knowledge, skills, abilities, and experience of organisations that use the model.

  • take a life cycle and continuous improvement approach.

There are prominent capability maturity models in the domains of cybersecurity and SOC operations, which are described below.

Capability Maturity Modelling (CyberCMM)

Cybersecurity Capability Maturity Modelling (CyberCMM) is globally acknowledged as a practical and successful approach to measure and improve the cyber resilience of enterprises. It helps organisations of all sectors, types, and sizes to evaluate and make improvements to their cybersecurity programs. It focuses on the implementation and management of cyber security practices associated with the information technology (IT) and operations technology (OT) assets and the environments in which they operate. A CyberCMM can:

  • enable organisations to evaluate and benchmark their cybersecurity capabilities effectively and consistently and prioritize actions and investments to improve their cybersecurity capabilities.

  • enable national bodies to share knowledge, best practices, and relevant references across organisations to help them improve their cybersecurity capabilities.

  • enable national bodies to carry out data-driven analytics from sectoral, cross-sectoral and trends-over-time perspectives.

Cybersecurity capability maturity models are typically designed and engineered to provide a self- assessment platform for individual CSEs. National bodies are developing central systems that aggregate data from individual CSEs to carry out data-driven analytics from sectoral, cross-sectoral, and trends-over-time perspectives.

US DoE C2M2

The US energy industry led the development of C2M2 to help organisations in the energy sector to assess their cybersecurity maturity and make optimum investments towards that end. The model can also be used by other organisations, irrespective of their size and domain.

The target sectors that have leveraged C2M2 include energy, critical manufacturing, government, healthcare, defence industrial base and financial services.

The C2M2 leverages a set of proven cybersecurity practices, focusing on both IT and OT assets and environments. The C2M2 domains are:

  • Asset Change & Configuration Management.

  • Threat & Vulnerability Management.

  • Risk Management.

  • Identity & Access Management.

  • Situational Awareness.

  • Events and Incident Response.

  • Third Party Risk Management.

  • Workforce Management.

  • Cybersecurity Architecture.

  • Cybersecurity Program Management.        

US DoD CMMC

The model was developed for the Dept of Defense by Software Engineering Institute, Carnegie Mellon University (SEI-CMU) in collaboration with the Johns Hopkins University Applied Physics Laboratory with the intent to protect sensitive national security information and to protect the Defense Industrial Base from frequent and complex cyberattacks.

The intent of the framework is to assess and improve the overall cyber resilience of the Defence Industrial Base (DIB). The cybersecurity capabilities of DIB organisations can be rigorously measured using CMMC. This can be used by the DoD to make risk-informed decisions regarding the information it shares with DIB contractors. The CMMC thus helps the DOD in establishing levels of confidenece with regard to the security of defense contractors and the DIB.

The CMMC framework draws on maturity processes and cybersecurity best practices from multiple standards as well as input from DIB entities and the DoD. It is primarily designed to secure the Defense Industrial Base (DIB) supply chain. The DIB organisations have to be certified by a CMMC third-party assessment organisation (C3PAO) at a particular level prior to bidding on contracts.

The CMMC defines the following:

  • a practice is a specific technical activity or activities that are required and performed to achieve a specific level of cybersecurity maturity for a given capability in a domain.

  • a process is a specific procedural activity that is required and performed to achieve a maturity level.

CMMC version 2.0 Dec 2021 defines 14 domains, three levels with practices for each level for the DIB, as under:

  • Level 1 – Foundational (level 1 of v1.0) - 17 practices 48 FAR 52-204-21 Oct 2016

  • Level 2 – Advanced (level 3 of v1.0) – mirrors NIST SP 800-171 (110 practices)

  • Level 3 – Expert (level 5 of v1.0) – based on subset of NIST SP 800-172.

The 14 domains are aligned with the families specified in NIST SP 800-171

  • Access Control (AC)

  • Awareness & Training (AT)

  • Audit & Accountability (AU)

  • Configuration Management (CM)

  • Identification & Authentication (IA)

  • Incident Response (IR)

  • Maintenance (MA)

  • Media Protection (MP)

  • Personnel Security (PS)

  • Physical Protection (PE)

  • Risk Assessment (RA)

  • Security Assessment (CA)

  • System & Comn Protection (SC)

  • System & Info Integrity (SI)

An organisation must meet both the process and practice level requirements to achieve that level of certification within CMMC.

IEC 62443 Maturity Levels

From an organisational perspective, the IEC 62443 standard series describes four “Maturity Level” ratings (ML 1 – ML 4). The ML rating system is used to evaluate how well an organisation defines and describes security processes and how well the processes are followed by the personnel involved. The maturity levels are distinct from security levels (SL-T).

CERT Resilience Management Model (CERT-RMM)

The CERT Resilience Management Model (CERT-RMM), developed by the CERT Division of SEI, Carnegie Mellon University defines 26 processes areas grouped into 4 categories - Enterprise Management, Operations Management, Engineering and Process Management. The resilience strategy translates to evolving measures to protect and sustain the assets viz. people, information, technology and facilities.

ReBIT Cybersecurity Maturity Model (CMM)

The RBI Cyber Security Maturity Model (CMM) is developed in Oct 2017 as an industry initiative coordinated by ReBIT. The CMM is closely aligned with RBI-CSF, which is harmonious with international standards, such as NIST CSF, COBIT 5.0, ISO 27000 and other standards.

RBI CMM focuses on i) implementation and adoption of the mandated cybersecurity framework uniformly in the financial firms, ii) understanding of the firm’s cybersecurity maturity in terms of the adoption of the regulatory cybersecurity framework, iii) benchmarking, and iv) regulatory tracking.

The CMM provides guidance through i) a methodical approach to measurement of risk, planning of controls and governance and security strategy execution to strengthen cybersecurity posture of financial firms, and ii) metrics-based treatment, benchmarking and prioritizing risk driven investment in security.

Secure Controls Framework’s™ (SCF)

SCF’s Cybersecurity & Data Privacy Capability Maturity Model (C|P-CMM) is an undertaking by SCF contributors to define maturity levels for the SCF’s control catalogue. The SCF leverages an existing framework, namely the Systems Security Engineering Capability Maturity Model v2.0 (SSE-CMM) and provides control-level criteria to an existing CMM model.

SOC-CMM

SOC-CMM, evolved from a research project to become a well-used standard for measuring capability maturity in Security Operations Centres. The SOC-CMM model was initially created to determine characteristics and features of SOCs, such as specific technologies or processes. It evolved to become the defacto standard for measuring capability maturity in Security Operations Centers.

At the core of the assessment tool lies the SOC-CMM model. This model consists of 5 domains and 26 aspects, that are each evaluated using a number of questions. The domains ‘Business’, ‘People’ and ‘Process’ are evaluated for maturity only (blue colour), the domains ‘Technology’ and ‘Services’ are evaluated for both maturity and capability (purple colour).

Maturity

SOC-CMM is a continuous maturity model, allowing improvements across all domains simultaneously and independently. The SOC-CMM uses maturity stages based loosely on the CMMI:

  • Non-existent. At this level, the aspect is not present in the SOC

  • Initial. The aspect is delivered in an ad-hoc fashion

  • Managed. The aspect is documented and delivered consistently

  • Defined. The aspect is managed using ad-hoc feedback on the quality and timeliness of deliverables

  • Quantitatively Managed. The aspect is systematically being measured for quality, quantity and timeliness of deliverables

  • Optimizing. The aspect is continuously being optimized and improved.

Capability

The SOC-CMM uses a continuous approach to measuring technical capability across the technology and services domains. These can be technical features, such as the existence of certain tooling options or other features such as service artefacts. Capabilities can be expressed at any maturity level. Just like with maturity scoring, capability scoring is continuous. Similar to the CMMI, the SOC-CMM supports 4 capability levels:

  • Incomplete. The capability is missing or lacking essential features

  • Performed. The capability is performed, but not standardised

  • Defined. The capability is deliverd in a standardised fashion

  • Managed. The capability is active managed and improved.

Methodology

The methodology used to create the SOC-CMM is a scientific research approach called Design Science Research. This type of research has a focus on bridging the gap between theory and practice and works well for areas that have not been extensively (scientifically) studied and clearly defined, as is the case for SOC capability and maturity. The goal of Design Research is the creation of a tangible result of the research effort. In this case, two artefacts were created: the SOC-CMM model, which is an abstract representation of SOCs and the self-assessment tool based on that model to evaluate capability maturity in a SOC.

The SOC-CMM self-assessment tool is available for download here.