Security professionals have access to the amounts of data never seen before. Antivirus software, firewalls, data loss prevention solutions – they all generate a staggering amount of alerts.
Security operation centres and the underlying SIEM technology allow us to aggregate, correlate and make sense of these vast troves of data. We can create dashboards and metrics that might look slick and even be useful to security teams but do such data add value to business stakeholders? Do they tell a story to the Board?
10,000 ‘attacks’ have been blocked. What does this mean? And, more importantly, so what? Are we reporting on what is easily available rather than what actually matters to the business?
Phishing simulations is another example. It’s understandable that we gravitate towards hard metrics that such exercises provide – we can track progress and count the number of people who clicked on a phishing email. Such metrics, however, can be counterproductive and I recommend proceeding with caution.
Most of the staff in your organisation are hired to do a specific job and that job is rarely to be a security expert. People are expected to click on links in the emails as part of their day-to-day activities. The aim of a security professional should be to build a trusted relationship with business stakeholders in the organisation. And tricking them into clicking on phishing emails might be counterproductive, especially if you are ‘rewarding’ them with follow-up training.
There might be a case for such exercises, but using a controversial template in an effort to demonstrate a high click rate is a step in the wrong direction. This can alienate your fellow co-workers, the people you are supposed to be helping.
This doesn’t mean you can’t select metrics that act as a proxy to your company’s level of cyber hygiene or security capability development. Knowing your audience and tailoring you message, however, is key.
You can indeed look backwards on the number of attacks, alerts, incidents and these can be useful in some contexts. Some would argue this acts as a justification for the investment in security tools that have been implemented to detect these intrusions. But this runs the risk of overwhelming the business with numbers that may carry little meaning or relevance to their concerns.
If you must communicate technical details that correlate with the maturity of cyber capabilities, I recommend forward looking metrics. For example, a number of systems with latest patches applied, number of systems scanned for vulnerabilities, number of systems with multi-factor authentication enabled. These will change over a specified period and can demonstrate increased coverage, maturity and trends. Measuring time between incident detection, response and recovery, alongside other parameters, for instance, can be a useful proxy for cyber resilience.
An approach like this can also help you inductively define risk appetite through discussions with the business on the desired parameters and associated risk reduction. Does the Board want you to move faster, increase coverage or shorten response times? If so, are they prepared to fund relevant initiatives or would they rather accept the associated risk?
The so what? question still needs to be answered here. I recommended tailoring your messaging to organisational objectives and business risks.
If available, check the company annual report: there is usually a section on business risks so it’s a good idea to align security risks to those in order to speak the same language.
Minimising operational risk associated with data breaches and regulatory fines is one way, enabling new business opportunities is another.
Cyber security can support launching new products and services, expanding to new markets, acquiring new entities and reducing insurance premiums, among other things. Know what business objectives are and align your security metrics to them to get buy-in and stay relevant.
The bottom line is, KPIs and metrics should be married to business objectives and expectations of the leadership. There is no one-size-fits-all approach to security metrics and they should be selected based on the organisational context. To be relevant, they must demonstrate the security function’s progress towards achieving the desired level of compliance, risk reduction and business enablement.
Cyber Security Leadership 
2 Comments