Metrics for Reporting on Continuous Monitoring

No matter how far along you are in establishing continuous monitoring, you should be reporting on the performance of the InfoSec compliance program and events that impact the organization’s position on security, compliance, or risk. While metrics are typically the preferred method for capturing relative information in reports to stakeholders — not all metrics are created equal. 

AuditBoard’s new ebook, The InfoSec Survival Guide: Achieving Continuous Compliance, explores how metrics fit into a strong continuous monitoring program. Download the full guide here, and continue reading to learn about how to formulate metrics that will best serve your compliance program. 

Strong metrics should measure the performance of established measures (e.g., controls, baselines, SLAs) across tactical areas impacting the business, allowing you to escalate red flags or concerning trends to executive management. These consist of Key Risk Indicators (KRIs), Key Performance Indicators (KPIs), and analysis. The design of your continuous monitoring program should support your level and frequency of reporting metrics. 

The following is a general overview of the metrics used in continuous monitoring reporting: 

1. KRIs. COBIT 5 for Risk defines KRIs as metrics that show the enterprise is or has a high probability of being, subject to a risk that exceeds the defined risk appetite. Because KRIs are pre-event metrics, they provide InfoSec the opportunity to engage with process owners and stakeholders before something becomes an issue. KRIs to consider tracking include:
   
   Number of critical systems or applications
   
   
   
   Number of users with access to all records (in a critical system)
   
   
   
   Number of competitor data breaches in the past year
   
   
   
   Number of malicious firewall events month to month
   
   
   
   Percent of third parties without a current security review
   
   
   
   Percent of incomplete security awareness training (or past due)
   
   
   
   Percent of systems without endpoint protection coverage
   
   
   
   Percent of systems with past due security patches
   
   
2. Number of critical systems or applications
3. Number of users with access to all records (in a critical system)
4. Number of competitor data breaches in the past year
5. Number of malicious firewall events month to month
6. Percent of third parties without a current security review
7. Percent of incomplete security awareness training (or past due)
8. Percent of systems without endpoint protection coverage
9. Percent of systems with past due security patches
10. KPIs. Key performance indicators are metrics that provide executive management and sometimes the board with a snapshot of how your security program is functioning over time. Use KPIs to illustrate a story about your environment when updating management. KPIs to consider tracking include:
    
    Compliance status by framework
    
    
    
    Control effectiveness by process
    
    
    
    Number of phishing emails reported, last campaign
    
    
    
    Number of phishing failures per quarter
    
    
    
    Percent of vulnerabilities remediated in/out of SLA
    
    
    
    Percent of completed policy acknowledgments 
    
    
11. Compliance status by framework
12. Control effectiveness by process
13. Number of phishing emails reported, last campaign
14. Number of phishing failures per quarter
15. Percent of vulnerabilities remediated in/out of SLA
16. Percent of completed policy acknowledgments 
17. Issues Management Effectiveness. KPIs specific to the performance of your issues management program are often what management is interested in most. Issue KPIs to consider tracking include:
    
    Number of new high-risk InfoSec findings
    
    
    
    Issues identified by root cause
    
    
    
    Issues that are repeatedly identified 
    
    
    
    Issues by department, owner, or system
    
    
    
    The trend of past due action plans
    
    
    
    Expiring risk acceptances
    
    
18. Number of new high-risk InfoSec findings
19. Issues identified by root cause
20. Issues that are repeatedly identified 
21. Issues by department, owner, or system
22. The trend of past due action plans
23. Expiring risk acceptances

To learn more about creating a strong foundation for reporting on continuous monitoring, download the full ebook, The InfoSec Survival Guide: Achieving Continuous Compliance.

About the authors

John Volles, CISA, is a Director of Information Security Compliance responsible for managing AuditBoard’s compliance, risk, and privacy obligations as well as helping customers understand AuditBoard’s security posture and position. John joined AuditBoard from EY, where he reviewed and implemented client compliance programs and supporting technologies. Connect with John on LinkedIn.