Crossing the Metrics Rubicon: Quest for the Perfect Measurement

Security metrics represent a great untamed wilderness for organizations trying to determine both their risk profile and the effectiveness of the resources they have allocated to their security program. When I first became a security person after a career managing customer service, finance, and operations teams, the most succinct argument I heard for the paucity of security metrics was that you cannot measure a negative. The thinking, it was explained, was you could not determine whether a security control was effective until it failed, at which point you could determine that it was ineffective and you needed to spend more money theoretically making it more effective until it failed again.

I suppose you could track mean time between failures as a metric to determine when it would next be likely you would petition the money masters at your organization in supplication. Of course this approach reminds me of the old comic strip where the boy, while riding in the family car, asks his father how the load limit on bridges is determined. The response is that they drive bigger and bigger trucks over the bridge until it collapses. The truck is then weighed and the bridge rebuilt exactly as it was before. The mom of course comments to the dad that he should just admit he doesn’t know – an admission many of us would hesitate to make when confronted with a question about how secure is our organization.

I hail from the school that you can measure most anything; unfortunately you may not always do it well. Once, when trying to impress his boss, my boss told him about the X million hits our firewall had taken that month. The boss’s response – so is that good? With that simple question he described the challenge we face with trying to provide metrics the business understands and can use to further strategic or tactical decision-making.

Post a Comment

Your email is never published nor shared. Required fields are marked *