Apple had more CVEs than any single MS product in 2015, but it doesn't really matter A count of the number of CVEs (Common Vulnerabilities and Exposures) issued on different platforms in 2015 has concluded that Apple was the most-advisoried operating system of the year, leading to gloating headlines that OS X is the “most vulnerable” of the lot. According to CVE Details, Mac OS X (all versions) apparently had …
PEOPLE don't love nonsensical numbers. MANAGERS love nonsensical numbers because it gives them the illusion that they're doing something when they can change the numbers. Advertisers also love them, because it helps them fool most of the people most of the time. But PEOPLE don't love nonsensical numbers, but it's unfortunately the only type of numbers we're being fed by the lying scumbags who fill the ranks of managers and advertisers.
The problem, perhaps, is that many (most?) managers have little or no understanding of the work that they're supposed to be managing. They're drawn from colleges with no more than an MBA to their name - rather from the ranks. And they've heard that measurability is good (and it is) but, since they don't understand their subject, they just grab desperately at any old number - no matter how meaningless.
"They're drawn from colleges with no more than an MBA to their name "
I think most of the MBAs that I've dealt with get their career first, then go to do an MBA whilst working, with their employer sponsoring the eye-watering fees. Many of them are past 30 and have been in "the ranks" for a few years.
Having got that out there, I still have the opinion that it stands for "Means Bugger All".
A quick "Common Vulnerabilities and Exposures or CVE's" at the beginning of the article would keep your readers from clicking to three different websites and reading that site's subscript to find out what the acronym CVE means. But apparently a CVE is common knowledge...
Counts of CVEs and the manufacturers assessments of risk level are the only stats we have to measure the performance in patching of software.
So what is required is a better analysis of the available data. With the limitations set out clearly.
Some patches may be trivial low risk issues to fix and others may cover gaping holes in systems but given the large numbers across a year, there might be sufficient info for broad trends to be shown.
Some companies publish their bugs as CVE's
Some companies would rather go to Chapter 7 that admit that their software has even one bug
Apple (and we love to hate them here) are actually pretty good at publishing their bugs a CVE's. Then you can see what ones have been fixed in an update.
As a software developer this simple act really helps find out if the bug is actually in my code or in the underlying OS. IBM is also pretty good at this as well.
but it does have a downside as it gives the Apple haters plenty of missile to sling at {cr}Apple.
CVE counts have been used for manufacturing headlines for well over a decade. Mr Chirgwin did point out that their are many problems with just taking the numbers without thinking, and points out some of the reasons why. As alien overlords appear to stop reading after a few sentences, I'll slip in a disclaimer about selecting a conclusion first and arrange the figures to match for the following flame bait:
Windows YYYY Server gets worse with each release: 2003 has 23 CVEs, 2008 has 149 and 2012 has 155.
Windows 8.1 comes with Internet Explorer, and MS Office is typcally installed for a total of 422 CVEs, putting it top of the list.
Last time I saw figures like these, a number was quoted for Linux by adding CVEs for each distribution. Ubuntu+Debian+Opensuse+Fedora is 422CVEs. Add in a few less popular distributions, and Linux becomes top of the list.
If the last one had you giggling, cvedetails have a chart of total vulnerabilities by vendor. Adding CVEs for all the versions of the top 50 MS products together gives 1590 CVEs.
The only frightening thing I can see about the alien overlord is he has not noticed that programmers are well aware of how these numbers are abused. Apparently someone has bought a bunch of articles adverts that take these numbers seriously. Writers for The Register know that the vast majority of commentards will not be impressed by such rubbish. Even Orlowski didn't try to run with this.
This post has been deleted by its author
>>"So, iTwats have gloated for years about how bug-ridden Microsoft was but now they top the chaart, the chart is meaningless."
Lot of downvotes for your post, but you're not entirely wrong. The article never actually says any of the things it suggests are true, it just throws a lot of doubt at it. Apple might not be worse because CVE numbers don't include severity and MS's CVEs could be more dangerous. Yes, but are they? Was any comparison between CVSS (severity) average on the two OSs done? Many CVEs are cross-platform. Sure, but wouldn't something like the PNG bug affect both? What is the reason for supposing that Apple is going to have more cross-platform bugs than Microsoft (would have thought it to be the other way round if anything). It pounces on the fact that CVEs are only recording reported vulnerabilities. Well of course they are. But is there any reason to suggest that MS is hoarding away vulnerabilities that they know are out there in the wild but never disclose? Probably not.
So it's really a FUD attack on CVEs. And I know some will reflexively downvote me for that, but it is -- there's no content actually showing that Apple aren't worse, but multiple arguments why they might not be. It does read a little like a pre-emptive attack on the numbers to show they don't matter. But for all their vagueness, it does show the number of vulnerabilities we know of occurring in the software in 2015 and Apple did score higher. And given MS produce a significantly wider breadth of software than Apple as well, that's worth paying attention to.
Apple users have long suffered from an illusion that their software is somehow inherently more secure. A wake-up call is well overdue, not a list of why you can go back to sleep.
How many bugs are being found and fixed internally? Companies can report those or not as they wish. It should make customers more willing to upgrade by giving them a better idea of what security issues those who don't upgrade will face, but it makes you look worse when people count CVEs.
If you look at Apple's CVE's they list who reported them, and a lot are listed as reported by Apple itself. Companies that don't air their dirty laundry will have fewer CVEs reported. One can argue either that Apple is being responsible and giving customers info they need even though it makes them look bad, or that the only reason they reveal this information is to scare their customers into upgrading to the latest OS.
Another issue is how good a company is at finding bugs and how quickly they create new ones. Let's say company X has 1000 undiscovered security bugs in their code and company Y has 2000. If company X spends 10x as much on tools, people and processes to identify/fix security issues and limit the creation of new ones maybe they fix 20% of them in a year and add only 50 new ones, while company Y fixes only 5% of theirs and adds 150 more. Company X will have a higher reported CVE count but has more secure code that is trending down in bug count while company Y's code is less secure and trending up in bug count. Unfortunately there's no possible way we could ever know these metrics for anyone...
Did the bulbs dimm when you entered the forum? The point the article was making is just how utterly pointless the CVE chart is for assessing the security of an OS or application, as stated on the website,
"Keep in mind that tech companies have different disclosure policies for security holes. Again, this list paints a picture of the number of publicly known vulnerabilities, not of all vulnerabilities, nor of the overall security of a given piece of software."
So a pretty pointless bow shot on the high seas of fanboi flame wars.
Sorry, but if your software relies on someone's else software to work, can't work without, and it is installed by default, any bug in that software is a bug of yours too.
From a user perspective I really don't care how a bug comes to my machine. If I install an OS, browser or any other application, and it has a flaw, I don't really care if it's in code you wrote or in libpng, OpenSSL or whatever - it was your decision to use such a library, not mine.
Almost but not quite. If the bug is in the underlying support system (OS, libraries etc) then it may well compromise your application. However if you know it is an external vulnerability you can potentially substitute libraries, work around a vulnerability or even shut down the system until the vulnerability can be addressed.
I can see all sorts of disagreement on each weighting classification, and endless debate on who runs the weighting definitions
Fortunately, we have a standard for that: CVSS 3.
It's not perfect, but it's widely used and widely accepted; and while people will certainly grouse about the scores of particular vulnerabilities, over a significant number of CVEs the aggregate score likely converges on something like a consensus among knowledgeable, relatively unbiased observers.
So, actually, something like the sum of the CVE x CVSS products for an organization would probably give you about as good a single-value metric for "total severity of reported security issues for the year" as you could get. Whether that's good enough to be useful is debatable, for all the reasons noted in the article and elsewhere.
One failing of such a metric is that some products have very few reported CVEs because, for cultural reasons, the people who tend to find vulnerabilities in them aren't inclined to report them. They're not particularly secure; they're just not part of that conversation.
I too would like to believe Cook is serious about product quality. But experience suggests otherwise.
Most of OS X is "barely good enough" rather than "rock solid." It's mostly petty annoyances - like Facetime taking a minute to work out you've picked up on a different device - rather than show stoppers, but they're still annoyances.
And I definitely know people who have lost photos and files because iCloud has creative ideas about syncing, and TimeMachine has creative ideas about backup robustness.
I really would like to believe that Tim Cook is serious about quality product.
So would I. Except the sales numbers are probably telling him that he doesn't have to be.
I think the release management is now back on track but, considering the lack of innovation, the number of bugs and the time it takes to fix them (compare Safari with Chromium), it's all a bit depressing.
Not sure why no one has mentioned this, but every CVE code is usually accompanied by a corresponding CVSS score, which would indicate impact (the 2.0 CVSS scoring isn't perfect, but it's useful).
Why not just re-run the numbers with anything with a CVSS score above 7 to get a more meaningful rating.
CVE's alone are garbage, you can declare them youself on any product for something as simple as no password lockout....
Here we go. Top 20 based on weighted average CVSS score:
9.6 Air Sdk
9.6 Air Sdk & Compiler
9.5 AIR
9.4 Flash Player
9.4 Office
9.3 Internet Explorer
9.3 Acrobat
9.2 Acrobat Reader
8.3 Firefox Esr
8.1 Thunderbird
8.1 Windows Server 2003
8 Seamonkey
8 Windows Server 2008
8 Windows Vista
7.9 Windows Xp
7.9 Windows 7
7.9 Windows 2003 Server
7.9 Itunes
Edit: Full list at https://kitd.github.io/CVEAnalysis.html
8.1 Windows Server 2003
7.9 Windows 2003 Server
Anyone running Windows Server 2003 is urged to switch to Windows 2003 Server immediately. (If you can't switch your entire deployment immediately, try to switch at least half your machines, to lower your average CVSS-weighted vulnerability to 8.0.)
"android" just gets a lot of publicity for vulns because of its "open" nature. The only issue for "android" is that some phones won't get patches. But the concerned consumer can purchase a phone that will. If you have an older phone you can use cyanogenmod and update it monthly. Its not that hard. We can argue about which OS is more buggy but they all have bugs. Smartphones are inherently safer than say a windows PC because of the vetted app stores and multilayered security models as well as being simpler. Also windows particularly is far more complex and therefore a large attack surface as well as a legacy of risky computing habits.
Flash recording 314
Which makes Adobe the undisputed kings of shite in this game IMHO.
Why? 'Cos Apple's candidates are Operating Systems, performing a wide variety of tasks with multiple points of exposure to the world.
Flash does only one thing and yet is of such appalling quality that it's got almost as many holes as most things that are an entire order of magnitude larger and more complex.
"As this chart (rather than the list favoured by most outlets) shows, Microsoft and Adobe both out-CVEd Apple for vulnerabilities “by vendor” across CVE Details' Top 50."
Really? http://www.cvedetails.com/top-50-vendors.php?year=2015
But of course, figures don't matter (unless it's MS, or Flash)
>>"PHP is included in OS X but the same bug was present on all platforms. Should that count as an Apple CVE?"
A CVE shows vulnerabilities anywhere that they are included. The CVE is therefore a CVE for both PHP and Mac OSX that includes that PHP code. (I'm guessing that you're talking about this, btw).
CVEs are focused towards the practical rather than the "fair". If your product has a vulnerability it doesn't matter if you can say it's not your fault or not, CVEs don't care - their for the customer's benefit. If I built a GNU/Linux distro that had lots of unmaintained packages included, my CVE count would be high, even though they were all other people's code. The same is true for everybody, btw. If something common to two vendors has an exploit in it, then that's +1 exploits to the count of both (and thus okay for comparison).
The problem here is it depends on how many versions of an OS a vendor releases and how they are counted.
For example, for every OS X point release (i.e. 10.10.1, 10.10.2, 10.10.3, 10.10.4, 10.10.5) then the CVE list is counting, say, a PHP vuln once for every release, so that's five exploits, right?
Note also there is a lot of overlap between iOS and OS X, so now we have a vuln that might also affect iOS 9.0, 9.0.1, 9.0.2, 9.1 so we should add another four to that list, so is that nine exploits for Apple?
Or is it one?
>>"Note also there is a lot of overlap between iOS and OS X, so now we have a vuln that might also affect iOS 9.0, 9.0.1, 9.0.2, 9.1 so we should add another four to that list, so is that nine exploits for Apple?"
The site doesn't differentiate between micro-versions, so iOS 9.0, 9.0.1, 9.0.2, etc. aren't going to rack up multiple counts for a vendor for the same issue. Though a vulnerability that was present in both iOS and OSX would of course count double so yes, there is a penalty for providing a broad range of software. Actually that's a count in MS's favour as they have 405 products listed on the database to Apple's 105. So if anything, the issue you highlight benefits Apple much more.
But the useful way to do comparisons, is by product. So for example you can compare Windows 8.1 with OSX:
http://www.cvedetails.com/product/26434/Microsoft-Windows-8.1.html?vendor_id=26
http://www.cvedetails.com/product/156/Apple-Mac-Os-X.html?vendor_id=49
You can see that 8.1 had 151 vulnerabilities in 2015 and OSX had 384 in the same period. That's why I called this article FUD. There's a very significant difference and the article makes no attempt to actually examine it, it just lists a lot of attacking questions in an attempt to dismiss the entire comparison - how do we know MS don't hide vulnerabilities? how do we know their vulnerabilities aren't more severe? what if Apple is being penalized for having the same vulnerability in multiple products? That's the essence of the article. There's no attempt to assess, only to discredit. As you can see from my response to your own post, it's actually not that hard to look into these questions and get a feel for whether or not the attack is justified. Instead the article simply does a pre-emptive attack trying to cast uncertainty and doubt on the findings.
No fear though, more trying to reassure if anything. So lets call it Reassurance Uncertainty Doubt (RUD) rather than FUD. These findings might not be what they look like (despite the fact that they probably are), so let's dismiss them.
Ever since Microsoft discovered that people actually believe statistics without investigating them it has been having a party with graphs and numbers (I know - I've seen them do this to whole governments and upper rank military). Being the evil sod I am, I suggested post-presentation fact checking, which quickly resulted in us no longer getting a copy of presentation post event for such efforts with sometimes frankly award winning excuses such as "it contains confidential information" (you just shared it with us).
Anyway, I don't have the time to check right now, but I hope the author accounted for creativity we encountered and defanged late last year.
Personally, I care less about the numbers. I care about the risk we're exposed to when running a normal IT operating environment, how easy/costly it is to stay ahead of the bad guys. That's why we no longer use Windows, and even if we did, there was no hope in hell we'd switch to Windows 10 - apart from the privacy risk, we also know what will happen when Microsoft has everyone on subscription. It will be like when they discounted education purchases: once all are locked in, the price will go up.
The only question is who will raise prices first this year: Adobe or Microsoft.
My money is on Microsoft.
>>Some brief samples showing level 10s / total:
>>MacOS - 46 / 384
If you want to limit it strictly to 10's, then Windows 8.1 has 3 / 196. So lower both in total and in number of maximum rated severities.
N.b. I picked Windows 8.1 rather than Windows 10 because the latter doesn't have a full proper year, yet so is not a fair comparison. I could have picked Windows 7 but that's an older version of their software. I think it is fairest to compare current version to current version, rather than old version to current version.
>>It came out on July the 24th and Windows 10 came out on the 29th so I'm just trying to understand your logic in excluding Windows 10.
You said "And how long has Yosemite been out?". That came out in October 2014. I'm excluding Windows 10 because it came out mid-2015 which would make it very bad for a benchmark of most CVEs in 2015.
Okay, people, double-check the math and we'll declare a winner. Let's say, by January 10. That way it's super fair. OS X (10!) and Windows 10. Or maybe we should use Windows 8.1 to be extra fair. Problem is that 10-8.1=1.9, and that's not an integer. We'd have to rectify that with the CVEs which are integers only IIRC.
Gimme a minnit to get my popcorn. This is gonna be good! :)
I'm predicting that Windows 8.1 will tie with OS X and Windows 10 will break the tie. Final score: 1.9. But, come on. Was there really any doubt?
/s
The numbers are sheer non-sense. Sometimes, a single CVE is created for multiple issues, just search for "multiple vulnerabilities" or click the link below to find out the extent of this issue.
https://www.cvedetails.com/google-search-results.php?q=multiple+vulnerabilities&sa=Search
This means that any information about count or average severity is just unreliable. Move on, nothing to see here...
One of the problems with Managers in general and especially with IT management is the mantra "you cannot manage what you cannot measure". No mention whatsoever about understanding what you are supposed to manage. If you are manufacturing widgets it is easy to measure output. If you are trying to keep an IT infrastructure running what do you measure that is meaningful? User satisfaction? Number of help desk tickets? Uptime or downtime? None of these are clear cut but managers love "metrics" so they will measure something, create pivot tables and charts, and send the meaningless reports to higher management.
I always thought MBA meant "Me Before All".
As a manager, I can tell you that those metrics are important. If I'm to convince the directors to give me £60,000p/a for a new full time Q&A person in my business unit, the first thing they're going to want is to see why I need that person. Saying "it will really help" or "please" unsurprisingly doesn't cut the mustard. The second thing they're going to want to see is the results. Again, saying "thanks, please keep giving that money every year" doesn't work. I need to be able to show something - a reduction in bugs in production, an increase in user satisfaction, a decrease in support tickets... something.
Sure, nothing is a perfect metric unless you're a Volkswagon emissions tester or something, but they are not "meaningless reports". Businesses are not built on trust. Money is not allocated on a whim. If you're running a team, you need to be able to communicate to the upper management in a language they can understand, what is happening and why.
Just because meaningless metrics are used to make decisions does not make them meaningful. Customer satisfaction? Most users do not respond to the surveys so how valid are the responses you do get? If your QA people didn't find any bugs last quarter does that mean you can assume your coders will never write buggy code again so your QA people are no longer needed? There are precious few valid metrics in this business and just because management wants them does not mean they exist. I had a manager who wanted dynamic DNS to work. After a lot of wasted hours of data collection and research I proved that with the domain and DNS configuration dynamic DNS was never going to reliable. He reviewed all the data and agreed; but, still wanted it to work so we kept using it and kept having to deal with production servers disappearing.
"If you're running a team, you need to be able to communicate to the upper management in a language they can understand, what is happening and why." This is the real problem, upper management is usually ignorant of what they are managing so middle management has to resort to shiny dash boards that look good but mean nothing.
Just throwing this one into the mix.
Weighting of CVE severity certainly makes the numbers a bit more sensible, but being devil’s advocate, perhaps to tell who has the worst record for code sloppiness, we should also factor in market share?
One of the key trends we have seen when it comes to vulnerability discovery and exploitation is that it correlates closely to the number of machines in the wild, and there is strong evidence to show causation.
Hence the term "security by obscurity". We all know that (with the exception of government spy agencies) hackers and virus writers only target systems with a large enough pool to make it worth their time. After all, 90% of malware is written for financial gain.
The MS platforms and Adobe Flash are obvious targets because of the sheer numbers, and the potential bounty that can be retrieved en-mass from the compromised machines. Hence, more beady eyes scrutinising the code for weaknesses.
I'm not a statistician, and have no talent with numbers, but in a very generalised manner, I can say that factoring this in would make Adobe's case look a little better, but would place OSX in a very poor light, indeed, given its very tiny market share.
On the other hand, this meteoric rise to vulnerability infamy for OSX could also be a short one?
Just like Windows XP back in 2003. The very sudden explosion in broadband connected machines meant a glut of vulnerabilities that had been dormant for years, (in the NT code) but were not exploitable in any practical way. Once exposed, Microsoft worked very hard, and quite successfully, to improve the security of their OS.
Perhaps a similar story is playing out with OSX? Until very recently, there was no real financial incentive to go looking for bugs to exploit in OSX, due to the very small numbers, but with the success of the iPhone, mac numbers have swelled dramatically, and therefore has become a viable target.
Maybe Apple will wake up and start taking security seriously? Maybe it'll take an iSasser worm to shake them out of apathy?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
