Code Spaces goes titsup FOREVER after attacker NUKES its Amazon-hosted data Source code hosting provider Code Spaces has suffered the ultimate cloud nightmare, having been effectively forced out of business by the actions of an attacker who managed to gain access to its Amazon EC2 control panel. The devastating incident began on June 17 when Code Spaces – a company that claimed to offer "Rock Solid, …
That's just utter bo**ocks. This has nothing to do with the cloud and everything to do with truly dreadful system administration. It could just as well have happened in a private data centre as on a cloud service. Cloud services may have their faults, but this is categorically not an example of one of them.
This sort of thing can happen in a data centre, but that's a problem with outsourcing in general. It's usually done because outsourcing is cheaper, and it's usually done cheaper because either staff are outsourced to india and they pay them peanuts or you discover that the reason they can provide it cheaper than you can in house despite using the same suppliers is that your in house solution had redundant discs in RAID and backups, and theirs didn't.
"Cloud services may have their faults, but this is categorically not an example of one of them."
Yeh, tell that to the users mate, I'm sure they will agree with you.
Vapourware and its data - one puff and it's gone. Wouldn't have happened with decent backups which aren't managed on-line. What a complete fail that is.
On the contrary, it is a good example. An internal data center wouldn't have had an easy to use WEB ACCESSIBLE front end you could use to cause all that damage.
All the crook needed was a single username/password to get in.
Plus, cloud pushes you to do everything on it, even when it's a bad idea. (See backups in the above article). This creates a single point of failure.
Sure, the administrators were idiots NOT to have backups elsewhere, but cloud helped push them in that position.
It does in the sense that an actual old-school IT server room would likely have some actual backup tapes or portable HDDs locked up in a cupboard, or trucked away to a physical (not virtual) off-site backup. When EVERYTHING is in the cloud, then it is susceptible to hackers in the cloud. When your backups are physical, then the hackers have to be physically present (unless they log into your UPS and cause it to burn the building down, only hours before it would have done so anyway).
Corrective action and lesson learned is multi-layered, including having non-virtual backups that are also available on the Internet.
"It does in the sense that an actual old-school IT server room would likely have some actual backup tapes or portable HDDs locked up in a cupboard, or trucked away to a physical (not virtual) off-site backup. When EVERYTHING is in the cloud, then it is susceptible to hackers in the cloud."
But it doesn't. Because EVERY business should have off site backups. It doesn't matter if you use the cloud, a datacenter, a colocation or your basement. Ate least ONE copy of the backups should be kept off site - and not accessible by the systems being backed up (ie: the backup can access the systems, to do a restore or a backup - but the systems can't access the backups).
Yes, the single sign on system that Amazon uses made it easier. No doubt about it. But the absence of off site backups... that was just neglect, not a cloud problem.
Even if they had substantial and appropriate backups, there still would have been massive disruption to their business and their customers. Arguing that it has nothing to do with the cloud ignores this point.
I think the other important point is the lack of layered security - a single login portal is surely a major flaw, though I don't know if that is 'cloud typical' or not.
"Even if they had substantial and appropriate backups, there still would have been massive disruption to their business and their customers."
Which is why you have a business continuity plan, which is a (tested) plan as to how you are going to continue the business come what may.
Note that in their explanation here: http://pastebin.com/WvtjMe9T
They state: Upon realisation that somebody had access to our control panel we started to investigate how access had been gained
It should, instead, have been: Upon realisation that somebody had access to our control panel we took down the website and disconnected all our systems from the internet..
Although that might be too much work for people working from home.
"Complete incompetent bastards. No decent backups, no disaster recovery plan, no business continuity plan, nothing nada.
inexcusable for an IT business."
They figured Amazon would handle everything.
They were wrong.
S**t happens. Guaranteed. It's not if you will attacked (as a successful business) it's when.
Code Spaces couldn't find $12.99 down the back of the sofa for a TFA key fob?
The only death involved with tape is people who aren't using it. Then again, these sort of people are the sort of inept muppets who don't change their tapes, or just leave the same tape in the drive to be overwritten constantly, so if these people had have been using tape then it probably have been written off along with everything else.
Cynical, moi?
Ok, I might be mildly paranoid, but I still I find it comforting to know that I have everything needed to recover from the worst possible disaster imaginable sitting off site and offline.
They had perfectly good backups - the backups didn't fail, someone with the authority to do so deleted them! They could've stored their backups in different locations across different servers but if you let someone get in and delete the backups, that is a process fail, not an infrastructure fail.
"They had perfectly good backups - the backups didn't fail, someone with the authority to do so deleted them! "
The backups DID fail. An off site backup is not "backup in another machine". An off site backup is "this backup of my data would survive, even if the whole datacenter got burned down in an attack".
This is an off line backup.
They could, for the sake of argument, have used Amazon to run the business - and another cloud provider (just to stay on the cloud) to keep the off line backups. I don't know. Azure? Google? RackSpace? Don't know, don't care - as the point is: it should be in ANOTHER company, not the same. And another location, of course.
"An off site backup is [something you can do a restore from]"
10/10.
Backups are no good if the restore might not work.
The guys/gals in this picture don't quite seem to have cottoned on to what backups are for.
Backups are so that you CAN do a restore. With some reasonable degree of confidence proportionate to the value of the data at risk, etc.
They had perfectly good backups - the backups didn't fail, someone with the authority to do so deleted them! They could've stored their backups in different locations across different servers but if you let someone get in and delete the backups, that is a process fail, not an infrastructure fail.
the main point is that true off-site backups cannot be accessed via any sort of wire or radio signal... off-site backups are exactly that... off-site... that means manually placed there in their fireproof box and manually removed from there when the next set is put in OR they are needed for disaster recovery...
"How lovely, so now disgruntled employee on his/her last day can take down whole company and permanently turn off the lights leaving the building. It's in the cloud folks, and it'll be there until wind blows it away."
And that is not unique to the cloud either. There are many accounts of disgruntled employees scheduling "dd if=/dev/zero of=/dev/sda bs=1m" to run on all in-house servers as they are leaving. Nothing new here. Plus, I've been called into to investigate a hack on in-house servers, where the attackers deliberately wiped Active Directory and IIS metabases on all servers. Because once you get into the domain, you get into all servers on the domain. That company also lost their entire business running on those servers, because while they had backups, reconstructing the configuration took two full days, by which time there were no customers left.
I doubt that there are that many accounts of disgruntled employees destroying their company when they leave. That is a criminal act and you go to jail for it.
And once you have a criminal record, you can say good-bye to any position higher than flipping burgers.
That's a high price to pay for a bit of disgruntling. There may be many people wishing that they could, but I really don't think there are that many who actually do it.
I doubt that there are that many accounts of disgruntled employees destroying their company when they leave. That is a criminal act and you go to jail for it
Sure, but the key question is how long would you go to jail if you plead guilty at the first hearing, and offered a clean previous record as mitigation? Only 1/3rd of whatever the sentanc was originally.
This becomes important because....
And once you have a criminal record, you can say good-bye to any position higher than flipping burgers.
Certainly you'd struggle to find work if you declared your conviction to an employer. If you didn't, and they failed to do appropriate background checks (smaller companies often fall down here), then you're fine.
Additionally, it appears the maximum jail time under the computer misuse act would be 10 years. Its exceptionally unlikely that you'd be given that as a starting tariff, so lets say 6 years. 1/3rd of that is 2 years. The rehabilitation of offenders act means you won't have to declare it after 4 years.
That's a high price to pay for a bit of disgruntling. There may be many people wishing that they could, but I really don't think there are that many who actually do it.
It is still a high price to pay, but it's not as high as it seems. You could readily fill a 4 year career break with a masters degree and a little backpacking holiday. Obviously, you have the 2 years inside to do as well, but it means you're only out of the game for 6 years total, rather than the rest of your days as you'd (rightly) expect for taking down your employer.
Certainly you'd struggle to find work if you declared your conviction to an employer. If you didn't, and they failed to do appropriate background checks (smaller companies often fall down here), then you're fine.
Not just small companies! I used to work for a financial services company (it was a subsidiary of a major UK bank) that had a new hire that was given to me to bring up to speed on the tech that we were using on his first morning. He was marched off the premises by security about two hours later only because someone in the office recognised him. He had done time (not for a tech-based misdemeanor) and then changed his name by deed poll and it wasnt picked up in the background checks. Presumably he forgot to mention his name change or time in chokey in the application. They might have increased their security checks now...
There are lots of inappropriate backgrounds checks as well. Many companies do them because everybody else is doing it and it's fashionable. I owned a small company for many years and employed dozens of people. I never ran any background checks and only sacked one employee for being a bad apple. I had suspicions about a couple of employees, but they did good work so what the hell?
I just told a company that was to hire me as an independent contractor to FO when they insisted on signed permission to do a full background check. I wan't going to be handling money in any way other than cashing the checks they paid me with, so why do they need my credit report? They weren't going to supply a company car or auto insurance, so why do they need my driving record? Their decision to use me was not based on having a degree, so why do they need my college transcripts? I wonder if they asked the plumber to sign those forms before doing any work or did they let him get on with unblocking the loo.
Aside from lying on his application, was the offense this bloke got sent up for in any way relevant to his employment with the company? I've never been handed an job application that asked about a name change. In the US, I think that one only has to fess up to felony convictions if asked.
"I doubt that there are that many accounts of disgruntled employees destroying their company when they leave. That is a criminal act and you go to jail for it."
Maybe.
On the other hand, the organisations that got (nearly) taken down frequently won't want their dirty laundry washed in public. When I've seen similar things happen (occasionally over multiple decades), none of them have involved criminal action, they've all been kept as quiet as possible.
I recall an episode from the 1970s or thereabouts - insurance company fired their tape librarian, gave her two weeks notice. She spent the next two weeks systematically erasing tapes that contained their entire database of customers, policies. They had no way to know who was a policyholder, what policies they had. They had to go back to their field agents all over the country and ask them to reconstruct the data from their own (paper) files.
Surely a call to Amazon could have resulted in a total lockdown once they knew they were under threat? A freeze of snapshots and an inability to remove history would have been a good place to be, if such things are possible in AWS...
Seems like they tried to fight the attackers single-handed and lost?
Shitty turn of events tho.. there's some right c*nts out there, sure enough..
Aye, and it isn't limited to a geographic distribution. I use the terms "cuntweasel" and "cockferret" rather often with zero overtones of misandry or misogony. They rank along side "douchepopsicle" and "gonadgremlin" in my lexicon. I don't care if some person or another takes offense beyond the obvious "this is an epithet". If they read deep hatred for $identifiable_group into that then it is entirely because of their own personal hangups.
I'm strictly egalitarian. I hate everyone equally, regardless of gender, race or so forth. Bunch of gonadgremlins, the lot of 'em!
@Jan 0
>He might be Australian. We use "c*nt" colloquially, the same way Brits use "bastard" and Americans use "asshole", and it isn't intended to be misogynistic.
And context should have told you that, Jan0. "cunt", as used by non-antipodeans, doesn't usually convey--and isn't ever intended to convey--'meany', 'bastard', 'bad person', 'Doctor Evil', etc., so your interpretation of its use as misogynist suggests that you perceive the word to be wholly appropriable, or to have a particular or singular use, and to have a narrow definition. I've found myself reminding a visitor from 'down there', though, that the word is used in these parts differently, when she was expressing anger at having had her bicycle stolen and referring to the thief as a "cunt". Different strokes...
"He might be Australian. We use "c*nt" colloquially, the same way Brits use "bastard" and Americans use "asshole", and it isn't intended to be misogynistic."
Indeed, but then, so do us Brits, and rest-of-world too!
It's not as if the poster was so annoyed with these people that he/she (Unlike Jan0, I'm not making a sexist assumption!) maliciously and nonsensically compared them to a feminine 'front-bottom'.
Rabid feminists do the legitimate cause more damage than sexist men
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
