The Role of Benchmarking in Cyber Risk Management

Organizations face an ever-increasing array of cyber threats, a huge catalog of disparate security technologies to deploy in response, and a plethora of industry best practices and standards intended to simplify their decisions on what to deploy. Given this complexity, benchmarking provides a valuable performance management tool, akin to a runner comparing their performance in training across multiple variables to their competitor’s performances.

Benchmarking, in its many forms, provides organizations with a powerful tool to assess their cybersecurity posture against peers, identify areas for improvement, and align their practices with industry standards. This blog post explores the vital role that benchmarking plays in effective cyber risk management, how it contrasts with performance management in other disciplines, and how it can help organizations enhance their security measures, allocate resources more effectively, and stay ahead of potential threats.

A High-Performance Toolset

Benchmarking is not unique to cybersecurity, but it is widely used in most other disciplines where high performance is needed. In essence, it is like a training tool for high-performance athletes. By comparing yourself to other athletes (organizations), you gain insights into best practices, identify areas for improvement, and set performance goals. You might notice top performers have invested in better shoes (advanced security technologies), hired specialized coaches (cybersecurity experts), or developed unique training regimens (robust security processes). Benchmarking allows organizations to compare the impact of these investments on their performance before the competition.

The Typical Approach to Cyber Risk Benchmarking

The most common cyber risk benchmarking relies on point-in-time maturity or control assessments, often coupled with subjective judgments. These assessments follow a structured approach to evaluating an organization’s cybersecurity capabilities across a domain or multiple domains, typically using a scale to categorize capabilities from initial or ad hoc stages to optimized or leading practices. 

A number of formal maturity assessments exist but differ considerably in their breadth and depth. For example, BSIMM is a widely recognized maturity assessment covering solely application security and related capabilities. Other maturity assessments utilize established standards, such as the NIST Cybersecurity Framework or ISO/IEC 27001, as the underlying framework for organizations to assess their cybersecurity practices. 

The Value of Benchmarking 

The value of these assessments extends beyond mere compliance or status reporting against these frameworks. They are crucial in improving cybersecurity planning by allowing teams to set concrete performance goals and eliminate vague objectives. This clarity facilitates more effective resource allocation and justification. In addition, by conducting widely adopted maturity assessments, organizations can not only track their progress in improving cybersecurity capabilities over time but also compare their performance against their peers. 

Crucially, there’s another vital aspect of benchmarking that often gets overlooked: the cost-performance ratio. In the high-performance athlete analogy, it’s not just about how fast you can run or how long you can endure—it’s about achieving optimal performance within your resource constraints. Just as elite running gear and professional training come at a premium, robust cybersecurity measures often require significant investment. However, not every organization has the budget of a Fortune 500 company, nor does every business face the same level of risk. Even without specific spending benchmarks, Benchmarking allows organizations to compare their performance to similar-sized organizations based on industry and other factors.

Benchmarking against Attackers is More Complex

However, let’s be clear – the modern cyber threat landscape presents a unique challenge: the real competition isn’t against industry peers. Imagine a marathon where runners are timing themselves against fellow teammates, analyzing their pace, endurance, and techniques. In this cyber marathon, organizations benchmark their security practices, measure their security posture, and compare performance metrics. But there’s a critical twist: while the runners race, they’re racing not just their peers but an ever-changing group of attackers trying to impede their progress or knock them out of the race entirely.

Moreover, the insights gained from benchmarking o, while valuable, often come with a delay. It’s like receiving performance data from last week’s training run to inform today’s race strategy. This lag is further exacerbated when benchmarking is performed only annually, as is common in many organizations. In such cases, decision-makers are essentially trying to navigate today’s threat landscape using a map that’s a year old. This significant delay means that while benchmarking can indicate where investments should be made to improve security posture, these insights may be woefully outdated when they’re acted upon. Organizations must, therefore, balance the historical perspective provided by benchmarking capabilities with near real-time insight into the necessary capabilities to deal with current threats.

The Need for Continuous Threat Informed Benchmarking

It seems obvious that a one-time or periodic assessment is insufficient for effective cyber risk management. Organizations must implement a continuous benchmarking process that includes both:

• Regular security posture assessments to identify gaps in expected capabilities
• Near real-time monitoring of key security metrics against industry benchmarks informed by threat intelligence

This near real-time insight can be sourced from threat intelligence and information sharing between industry peers and security alliances. This collaborative approach allows companies to identify key metrics that can help benchmark their capabilities against current threats. Some examples of these metrics could include:

• Mean Time to Detect (MTTD) incidents measure how quickly an organization can identify a security breach or threat.
• Mean Time to Respond (MTTR): This indicates how rapidly a company can react to and mitigate a detected threat.
• Mean Time to Patch: A proactive measure that shows how quickly an organization can apply critical security updates.
• Percentage of identities with Multi-Factor Authentication (MFA) enabled: This metric reflects the strength of an organization’s access controls.
• Attack surface measurements: These quantify the total number of potential entry points for attackers.
• Data blast radius: This metric estimates the potential impact of a data breach by measuring the scope of data that could be exposed in a single compromised account or system.

By comparing these metrics with threat intelligence data and information shared by peers, organizations can better understand their ability to withstand current threats. This approach allows for a more dynamic and relevant benchmarking process that goes beyond simple comparison with industry averages.

Benchmarking should also focus on an organization’s preparedness for unforeseen cyber threats compared to industry best practices. This involves conducting regular scenario planning and tabletop exercises to test readiness and then comparing results to industry benchmarks. Organizations should stress-test their security measures against advanced persistent threats and evaluate their performance against top-performing peers. 

Maintaining a Cost-Effective Security Posture

Remember, in cyber, the goal isn’t just to keep pace with your peers—it’s to outrun the attackers. Effective benchmarking provides the insights and direction needed to stay ahead in this never-ending race while optimizing resource allocation and maintaining a cost-effective security posture.

In conclusion, benchmarking in cyber risk management is a powerful risk management tool when used comprehensively and continuously. It allows organizations to not only measure their high-level performance against peers and industry standards every now and again – but to continually improve their security posture, adapt to new threats, learn from the best, and prepare for the unexpected. By embracing this holistic view of benchmarking, organizations can develop more robust, resilient, and efficient security strategies, ultimately leading to more effective and sustainable risk management practices in our increasingly digital world.