BOSTON — The security industry has done a poor job of finding ways for companies to measure their security, but that does not mean that collecting data is not valuable, the former head of the U.S. Department of Homeland Security's cyber group told attendees at the SOURCE Boston conference on Thursday. Amit Yoran, CEO of security …
The problem with risk management
Fundamentally, the whole metrics racket is a hiding to nothing when it comes to security. Suppose you have a webserver, and for the ease of the example, let's say your company does all it's business through this system. It has a bug. Should you patch it? Well, you can put a number on amount of business lost through downtime when patching (yes yes, no clustering here, we're in Gedankensville, OK?) You might even people to put a wet finger in the air and come up with some sort of guesstimate about how much it would cost you in lost business and reputational damage should you arrive at work one day to find your front page replaced by Fleshbot (although I'd argue that value is entirely theoretical, and anyway is probably a hell of a lot less than you might like to think it is - especially if you're the person trying to diddle the numbers so you get a bigger budget next year.) One thing you absolutely /cannot/ do, though, is put any sort of probability value on the chances of getting pwned /through that specific vulnerability/, per day. That's the sort of numbers the insurance business like to crunch to work out your car insurance premium, and why middle-aged me pays less to insure my 250 BHP turbo-nutter-bastard mobile than a 22 yo with a hot hatch, set of alloy wheels, Haynes manual and a bodykit :) )
Given that the final number at the bottom of the page that's supposed to allow you to rank your systems, the stuff you could do to secure them, and how much you should spend to do so are based on garbage - and we've all seen what lousy risk management based on garbage input can do to the world economy - it follows that one should beware of snake-oil salesmen bearing metrics.
Mine's the one with the "kick me" note stuck on the back by the Sales Director...
"He who would defend everything, defends nothing.!"
Gneisenau (I THINK!)
"He who would collect all data copllects nowet!
Gary (with appologies to Gneisenau!)
"[...] Former head of the U.S. Department of Homeland Security's cyber group [...]: 'we need to measure everything'"
Why am I not surprised?
Re: need to measure everything
But isn't the real problem that what you are actually trying to measure is what you are NOT doing, so that you can identify it and do something about it. Of course if you could measure it, then you would know about it, and knowing about it you would have done something about it.
Or not, depending on your local implementation of the PHB.
- iPad? More like iFAD: Now we know why Apple ran off to IBM
- +Analysis Microsoft: We're building ONE TRUE WINDOWS to rule us all
- Climate: 'An excuse for tax hikes', scientists 'don't know what they're talking about'
- Analysis Nadella: Apps must run on ALL WINDOWS – on PCs, slabs and mobes
- Major problems beset UK ISP filth filters: But it's OK, nobody uses them