Ponemon Study Shows Security Metrics May Not Be Understood By Management

A new joint study between the Ponemon Institute and Tripwire suggests that risk-based security metrics may be too complicated for many senior members of the management team to understand.

A survey of 1,321 security professionals from the UK and US discovered that 75% thought that metrics were important to a risk-based security program.

Far more surprising, perhaps, was the discovery that over half of the respondents (53%) didn’t feel that the metrics being used in their company were properly aligned with the organisation’s business objectives.

Additionally, 51% of those questioned were unsure whether the metrics being produced were fully understood by senior execs.

“You can have all the right numbers for anything but C level needs real context re biz impact to drive home msg.”
James Lee

There were several reasons reported as to why metrics were not proving to be as effective as they could be and each are rather concerning in my opinion.

Perhaps the most worrying response was that 18% of senior executives were not interested in the information. If that is the case then one would have to wonder why the business is investing in metrics in the first place?

“Key issue is getting them to associate metrics data with a tangible risk to them (ideally personally).”
Daragh O’Brien

Another area that risk-based security professionals should consider is that 59% of the survey respondents said that the metrics they were producing were too technical to be understood by management who themselves were non-technical. Is that a management failing or does it suggest that the metrics themselves need to change? Or perhaps the way in which the metrics are presented to senior executives needs to be re-evaluated?

“Key issue is translating technical metrics into something the business cares about. Context is king.”
Brian Honan

Also of note is the fact that 40% of those surveyed said that they only communicate actual incidents to executives. Surely risk-based security management should be more of a proactive discipline?

Other reasons given for not creating metrics that were understood by senior executives showed that 48% said that more pressing concerns took priority and 35% said that the preparation and reporting of metrics was too costly in terms of time and resources.

“It needs to have a direct positive impact to their business (and bonuses :p)”
Filip Maertens

I asked Tripwire’s Community Engagement Coordinator Anthony M. Freed what he thought security professionals could do to improve the way in which they present technical information to non-technical executives and he said,

“One of the contributing issues may be that the security team has not made the effort to tailor the metrics in such a way as to make them compelling in that they are directly tied to the organization’s primary business objectives. Simply counting and categorizing event types in a effort to demonstrate how many attacks were ostensibly prevented does not paint a picture for management about the true impact of security operations.

While those events need to be documented, they don’t necessarily have to be shared in such raw form. Instead, perhaps the security team should characterize the events in relation to the potential impact they could have on operations, business continuity, intellectual property, and ultimately brand reputation, depending on the nature of the business.

For example, if your team can show that there was an increase in events targeting servers that contain sensitive PCI data of customers, and the company is a large retailer, and the team shows how they were able to adapt rapidly to the increase in attempts to punch holes in the systems, and how their efforts resulted in the attackers for the most part just moving on to find lower hanging fruit, that is something that would resonate with an executive.”

How do you approach risk-based security metrics within your organisation and what challenges do you face in presenting them to the senior management team?

Managing Information Security with ISO 27001

course.gifIn partnership with the Centre for Software Engineering we are running a two day course on “Managing Information Security with the ISO 27001 Information Security Standard”.  The course is scheduled to run on the 20th and 21st of October 2008 and you can book your place on the course by contacting the Centre for Software Engineering.

The “Managing Information Security with the ISO 27001 Information Security Standard” course provides a framework that will enable those responsible for securing sensitive information assets using a quality based approach to identify key assets and how best to manage the associate threats and risks.The subjects covered include:  


  • Overview of information security
  • Introduction to the ISO 27001 Information Security Standard
  • Identifying key information assets
  • Identifying risks
  • Strategies for mitigating and managing risk
  • Implementing appropriate security controls
  • Monitoring the effectiveness of security controls.

The course materials are supported with a number of practical exercises, tips and case studies to illustrate and give experience in applying the techniques described.  More details of the course are available from the Centre for Software Engineering


Call for More Companies to Certify to ISO 27001

Michael Brophy, CEO of Certification Europe, makes a strong argument for companies here in Ireland to certify against the ISO 27001 Information Security Standard, especially in light of the recent data breaches in Bank of Ireland.  Michael is quoted in both The SiliconRepublic.com and in Saturday’s Irish Independent on the frustrations he feels regarding the lack of take up of the ISO 27001 Information Security Standard here in Ireland.

Michael highlights that in a number of other countries such as the UK, companies are obliged to comply with the ISO 27001 Information Security Standard.  He points out that in the UK all financial institutions have to meet the standard or otherwise the UK’s payment association (APACS) will not deal with them.  In Ireland Michael states the only financial organisation certified against the standard is a Credit Union in Waterford.  Most companies certified in Ireland are telecoms companies, data-centres or pharmaceutical companies.

I have to agree with Michael.  Too many organisations are paying lip service to information Security and are not investing the time or resources necessary to ensure the security of confidential information.  Businesses tend to think that information security is a technology problem and therefore there is a technical answer to it.  That simply is not the case.  Information security is a business issue and one that cannot be left solely to the IT department to deal with.  Not that IT people are not competent to deal with certain aspects of information security but the responsibility for something as important as security should be with senior management within the business.

One of the key requirements of the ISO 27001 Information Security Standard is that there must be senior management buy in.  This is not merely a signature on a project charter but evidence of ongoing involvement is required such as senior management signing off their acceptance for the risk management approach undertaken by the organisation. 

By implementing ISO 27001, organisations can demonstrate independent third party verification that their Information Security Management System meets an internationally recognised standard.  This provides a company, its staff, customers and partners with the confidence that they are managing their security in accordance with recognised and audited best practises. 

By adopting the risk and standards based approach to implementing an Information Security Management System in accordance with ISO 27001, companies can reap many advantages, not least being better able to demonstrate compliance with legal and industry regulatory requirements.

ISO 27001 is not difficult and indeed need not be expensive.  You can download a PDF copy of the standard for US $30 from the American National Standards Institute’s website.  By complying with the standard you can have confidence in your ISMS without having to seek certification.  Depending on your internal resources you can also run the project internally using the guidelines provided in ISO 27002.  Of course if you need external guidance and assistance we would be happy to discuss your requirements.

Insuring Against Hack Attacks

The Canadian based insurance firm Executive Risk Insurance Services announced that they are launching a new corporate insurance product enabling clients insure against the associated costs of an information security breach.  The new insurance plan will not only cover the costs of the actual damage caused against systems by an attack but also the additional costs of notifying affected customers, compensating credit card companies or putting in place credit monitoring.

This is not the first insurance company to enter the world of cyber attack insurance, Chubb and the American International Group Inc. already offer products in this area.  Insurance companies are not known for taking risks, in fact they are well versed in identifying and estimating risks.  So what is significant is that these companies see a gap in the market and they will be no doubt followed by others.

As companies become more aware of the threat posed to them by their exposure to cyber attacks the boards of these companies will look at ways to minimise this exposure, especially when those on their payroll responsible for information security cannot gaurantee 100% security.  Insurance policies are used by many organisations to plug gaps in how they manage their exposure to certain risks.

The involvement of insurance companies in the field of information security will force a number of changes in the industry.  Just as in the physical world where businesses have to invest in certain security solutions that meet standards and are managed and supported by appropriately trained personnel in order to get insurance coverage, we will see the same thing happen in the area of information security.   In order to reduce their insurance premiums companies will be forced to better understand their risk exposure, implement proper security controls and ensure that their staff are appropriately trained.

What the growth of cyber insurance will hopefully do is bring information security into the boardroom where it will become a business issue which needs to be managed like any other business risk.

Upcoming Event – "Emerging Information Security Threats & Solutions"

As part of Trigraph Professional Services‘ IT Security Series, Brian Honan will speak at the upcoming “Emerging Information Security Threats & Solutions” briefing event scheduled for January 28th 2008.  This Briefing provides managers and key decision-makers in organisations with an overview of the latest threats posed to their organisation while also outlining ways to reduce these threats against critical organisational assets.  More details on the briefing is available from Trigraph Professional Services’ website

An Overview of Information Security Standards

Over the years numerous people have asked me various questions about Information Security standards.  In the main I get asked the same questions.  I thought it would be a good idea to try and summarise them here for others to benefit from. 

Can you explain what a security standard is?

A security standard is like any other standard within any other industry.  A standard is “a published specification that establishes a common language, and contains a technical specification or other precise criteria and is designed to be used consistently, as a rule, a guideline, or a definition”. Further, according to ISO, standards “contribute to making life simpler, and to increasing the reliability and effectiveness of the goods and services we use”.

In essence a standard is a common set of rules, definitions and agreed “regulations” that all parties can refer to for common reference.  A standard would be a set of minimum requirements that an organisation must meet in order to claim to be compliant with the standard.

Why do we need standards?

Standards provide us with a common set of reference points to enable us to evaluate whether an organisation has processes, procedures and other controls in place that meet an agreed minimum requirement.  If an organisation is compliant/meets a certain standard then it gives third parties such as customers, suppliers and partners confidence in that organisation’s ability to deliver to that standard.  It can also provide an organisation with a competitive advantage over other organisations.  For example an organisation that is compliant with a security standard may have an advantage over a competitor who does not when customers are evaluating their products or services.

In other cases certain regulatory and legal requirements may specify certain standards that must be met.  For example if your company processes credit cards then you must be compliant with the PCI DSS Data Security Standard.  This standard is a standard specified by the major credit card companies such as VISA & Mastercard.  If you are not compliant with this standard then you can either be fined, face higher processing charges or indeed those credit card companies may refuse to do business with you. 

In addition if you are meant to be compliant to a standard but are not and suffer a security breach then you could face potential law suits from those customers impacted by that breach.  TJX, the parent company of TK Maxx, suffered a security breach resulting in over 45 million credit card details being accessed by hackers.  TJX was meant to be PCI compliant but was not and is now facing lawsuits from impacted customers.

Standards can also help organisations meet with regulatory requirements such as the Data Protection Act, SOX, HIPAA etc.  By using a standard to create a strong foundation for managing and securing your systems you will find it easier to meet existing and new regulatory requirements easier than an organisation that does not.

Can you tell me more about ISO 27001?

The following  excerpt from a pevious Blog post titled “Why use ISO 27001?” provides more details on ISO 27001;

ISO 27001 is a vendor and technology neutral internationally recognised standard which provides companies with a risk based approach to securing their information.  It provides organisations with independent third party verification that their Information Security Management System meets an internationally recognised standard.  This provides a company, and its customers and partners, with the confidence that they are managing their security in accordance with recognised and audited best practises. 

However, in my opinion companies that have implemented an ISO 27001 based ISMS can demonstrate many efficiencies and other benefits such as;

Increased reliability and security of systems:
Security is often defined as protecting the Confidentiality, Integrity and Availability of an asset.  Using a standards based approach, which ensures that adequate controls, processes and procedures are in place will ensure that the above goals are met.  Meeting the CIA goals of security will also by default improve the reliability, availability and stability of systems.

Increased profits:
Having stable, secure and reliable systems ensures that interruptions to those systems are minimised thereby increasing their availability and productivity.  In addition to the above, a standards based approach to information security demonstrates to customers that the company can be trusted with their business.  This can increase profitability by retaining existing, and attracting new, customers.

Reduced Costs:
A standards based approach to information security ensures that all controls are measured and managed in a structured manner.  This ensures that processes and procedures are more streamlined and effective thus reducing costs.

Some companies have found they can better manage the tools they have in place by consolidating redundant systems or re-assigning other systems from assets with low risk to those with higher risk.

Compliance with legislation:
Having a structured Information Security   Management System in place makes the task of compliance much easier.

Improved Management:
Knowing what is in place and how it should be managed and secured makes it easier to manage information resources within a company.

Improved Customer and Partner Relationships:
By demonstrating the company takes information security seriously, customers and trading partners can deal with the company confidently knowing that the company has taken an independently verifiable approach to information security risk management.

ISO 27001 can be implemented within an organisation as a framework to work against or indeed the organisation can seek to gain certification against the standard.

What kind of security standards are available?

There are numerous standards available.  These can be broken down into three main sections;

  • Business Standards
  • Product Standards
  • Individual Standards

On my Blog the post “List of Security Certifications”  outlines all the certifications that I am aware of within the information security industry.  As you can see they are many and varied.

You can be assessed and certified against any of the above to demonstrate that you meet the minimum requirements to satisfy the standard.  If you meet those requirements then you can be certified against that standard. 

So a business standard would apply to an organisation and state they meet the requirements for the organisation to satisfy the standard.  Product standards mean when you purchase a product you know it has been independently accessed as being secure according to a predefined criteria.  If you are hiring someone as a member of staff or as a consultant you can determine if they have the minimum knowledge that you require for that role by looking at the standards the have earned. 

The following post on my Blog give some of my thoughts on certification schemes

How can we obtain the standards?

In order to obtain a standard I suggest you;

  • Determine which one is suitable to you and/or your organisation or product.
  • Become familiar with that standard.  You can obtain a copy of that standard from the organisations who develop the standard or it may be available from other third parties.
  • Engage someone with knowledge of that standard, either in-house use an external consultant.
  • Determine what gaps currently exist within your organisation against the standard and develop a plan to address those gaps.
  • Engage with a certification body to achieve the standard.

Is there a difference between security standards?

Yes there are differences.  Some are more respected than others, some are more stringent than others.  This is especially so in the individual certifications/standards where some of them would be seen as entry level qualifications.

How do standards get implemented?

The normal process to meet a standard goes along the following lines;


  • Implement the standard.
  • Engage a third party to audit you against the standard.
  • That third party determines if you meet the standard and whether or not you achieve certification against the standard.


  • Select the standard you wish to achieve.
  • Submit your product to the company authorised to test your product against that standard.
  • Have your product tested and if passed it will be certified (note that this can be a very costly exercise)


  • Select the standard/certification you wish to achieve.
  • Study against the requirements.
  • Sit an exam
  • Pass the exam.  Some certifications require verifiable work experience in the field on top of passing the exams.

What does it cost to implement a standard?

That can depend.  In most cases the biggest costs is in the time and people involved in trying to get the standard.

Does it make a difference if you are a small business or large corporation when you put security standards in place?

It makes no difference.  The standards apply to all companies of all sizes.  In some cases it may be wise to implement a standard when the company is small so the standard is ingrained as part of the culture of the company.  Often big companies may also have to “re-educate” themselves in how to do things in accordance with the standard and break bad habits that may be in place already.

What happens if you don’t have security standards in place?

Not having security standards in place may have the following implications;

  • If you need to be compliant with certain standards, e.g. PCI DSS, then you may face financial penalties and also loss of business.
  • You may find it more difficult to meet new regulatory and legal requirements as you may have to “reinvent” the wheel for each of these requirements, whereas complying with a standard can give you a solid foundation to meet these new requirements.
  • You may lose business to competitors that are compliant with the standards as they may be viewed as being more reliable by potential customers.

Do all businesses need them?

It depends.  For example, if you operate in certain industries then you may need them or if you process credit cards you need to be compliant with the PCI DSS standard.  In general though it would be viewed as good business practise for your company to be compliant with a security standard, similar to your company being compliant with the ISO 9000 quality standard.

What can potentially go wrong with your security standard?

The biggest problem is paying “lip service” to the standard.  This often happens if companies simply go for the standard for a marketing exercise or simply just to achieve the standard.  This then results in what I call “Tick List Security”. 

Tick List Security is where a company just implements security controls simply to meet a certain standard.  The company does not really care about being secure but simply wants to tick all the boxes on the requirements to meet the standard.  This can be a dangerous play as the organisation thinks they are secure but in reality they are not.

In my experience companies that go for standards for solid business reasons such as improving their processes, procedures and ultimately their security tend to be more successful and get more benefit from the exercise.

The other issue I often see if companies not maintaining their required documentation and record keeping for the standard.

How often do they have to be updated?

That depends on the standard and on your requirements.  If you achieve ISO 27001 you have a series of continuous audits to ensure you are still compliant with the standard.  From time to time the bodies setting the standards may also update/change the standard to keep them in line with the modern environment.

Where can you find out more about security standards and how do you find the one which is right for your business?

Most of the standards are available from the bodies that determine them and in many cases there are third party websites available to provide more guidance and information.

What is involved in being audited against a standard?

Dr. Gary Hinson, founder of Global Security Week and owner of the NoticeBored Blog, has an excellent “Frequently Avoided Questions About IT Auditing” page on his website.  Gary does more justice to this than I possibly could.

I hope my above thoughts offer some insight into the world of information security standards.  I would be very interested to hear your own thoughts and experiences regarding standards.

NIST Releases Three New Special Publications

The US National Institute of Standards & Technology (NIST) have just released three new special publications.  They are;

SP 800-111 Guide to Storage Encryption Technologies for End User Devices.

SP 800-114 User’s Guide to Securing External Devices for Telework and Remote Access.

SP 800-115 Technical Guide to Security Testing.

I have found the material NIST produces to be excellent reference material and always worthwhile reading.  While I have not yet had a chance to read any of the above yet I know what will be on my reading list over the next few days.

ISF Releases The Standard of Good Practise

The Information Security Forum have released the latest version of the Forum’s “The Standard of Good Practise”.  This is an excellent resource for anyone tasked with identifying controls to improve the security of the information and systems in their charge. 

“The Standard of Good Practise” is broken down into the following key sections;

  • Security Management
  • Critical Business Applications
  • Computer Installations
  • Networks
  • Systems Development
  • End User Environment

At over 372 pages it is not a light read but well worth the time to become familiar with.

For a list of other security standards and certifications check out our earlier posting on the List of Security Certifications.

Enterprise Ireland Podcast on Security Now Available

Enterprise Ireland run a series of podcasts aimed at SMEs to help them better understand some of the issues they face with regards to deploying and managing technology.  Brian Honan was invited to take part in the latest podcast to discuss IT security and how it can be addressed by SMEs.  SMEs face the same security challenges that larger organisations face but often with fewer or indeed no resources inhouse.  The other speaker invited to attend is Mike Harris, Director in Ernst & Young’s Risk Advisory Services practice.

In this podcast Brian and Mike discuss some of the key challenges facing SMEs and provide pointers on how to address those challenges.  Issues discusses include;

Spam Issues
In-House IT & Outsourcing
Instant Messaging
Viruses, Zombies and protecting SMEs
Customer Data
Setting policies & the Data Protection Act
What should SMEs Address?

The podcast is available at Enterprise Ireland’s OpenUP website

Don’t forget the “Computer Security Assurance Checklist” which is designed for managers as a checklist to determine if their IT Security is being addressed and is available from the BH Consulting website.