A new joint study between the Ponemon Institute and Tripwire suggests that risk-based security metrics may be too complicated for many senior members of the management team to understand.

A survey of 1,321 security professionals from the UK and US discovered that 75% thought that metrics were important to a risk-based security program.

Far more surprising, perhaps, was the discovery that over half of the respondents (53%) didn’t feel that the metrics being used in their company were properly aligned with the organisation’s business objectives.

Additionally, 51% of those questioned were unsure whether the metrics being produced were fully understood by senior execs.

“You can have all the right numbers for anything but C level needs real context re biz impact to drive home msg.”
James Lee

There were several reasons reported as to why metrics were not proving to be as effective as they could be and each are rather concerning in my opinion.

Perhaps the most worrying response was that 18% of senior executives were not interested in the information. If that is the case then one would have to wonder why the business is investing in metrics in the first place?

“Key issue is getting them to associate metrics data with a tangible risk to them (ideally personally).”
Daragh O’Brien

Another area that risk-based security professionals should consider is that 59% of the survey respondents said that the metrics they were producing were too technical to be understood by management who themselves were non-technical. Is that a management failing or does it suggest that the metrics themselves need to change? Or perhaps the way in which the metrics are presented to senior executives needs to be re-evaluated?

“Key issue is translating technical metrics into something the business cares about. Context is king.”
Brian Honan

Also of note is the fact that 40% of those surveyed said that they only communicate actual incidents to executives. Surely risk-based security management should be more of a proactive discipline?

Other reasons given for not creating metrics that were understood by senior executives showed that 48% said that more pressing concerns took priority and 35% said that the preparation and reporting of metrics was too costly in terms of time and resources.

“It needs to have a direct positive impact to their business (and bonuses :p)”
Filip Maertens

I asked Tripwire’s Community Engagement Coordinator Anthony M. Freed what he thought security professionals could do to improve the way in which they present technical information to non-technical executives and he said,

“One of the contributing issues may be that the security team has not made the effort to tailor the metrics in such a way as to make them compelling in that they are directly tied to the organization’s primary business objectives. Simply counting and categorizing event types in a effort to demonstrate how many attacks were ostensibly prevented does not paint a picture for management about the true impact of security operations.

While those events need to be documented, they don’t necessarily have to be shared in such raw form. Instead, perhaps the security team should characterize the events in relation to the potential impact they could have on operations, business continuity, intellectual property, and ultimately brand reputation, depending on the nature of the business.

For example, if your team can show that there was an increase in events targeting servers that contain sensitive PCI data of customers, and the company is a large retailer, and the team shows how they were able to adapt rapidly to the increase in attempts to punch holes in the systems, and how their efforts resulted in the attackers for the most part just moving on to find lower hanging fruit, that is something that would resonate with an executive.”

How do you approach risk-based security metrics within your organisation and what challenges do you face in presenting them to the senior management team?