The cloud floats majestically away...
Beijing is going to eat the West for lunch.
Microsoft has managed to repair its Windows Azure cloud, after an expired SSL certificate downed storage and other services for people across the world. Ninety-nine percent of the affected services have been brought back online, Redmond said early in the hours of Saturday morning, Pacific Time. "We will continue monitoring …
Beijing is going to eat the West for lunch.
Yes. But because they're using chopsticks it will take a while.
To err once is human.
To err twice takes a super-mega-corp.
It's a certificate issue... Nothing to do with windows or linux
This time only with Microsoft, not Windows.
This shouldn't surprise us. Their software is absolute shite - what in the world would make us think they were competent to run a chook raffle, let alone an online service?
IBM's "state of the art" Cloud data centre in Auckland was completely down for 30 hours+.
No public expanation.
(after a while the newspaper shut up about how long it was, and just started tell us how many racks it had, and how many backup generators. Edvertorial anyone?)
This is a classic example of an operations-management failure. It's got nothing to do with the OS. And if you think only Microsoft has failures, you probably missed hearing about when Amazon's load balancing system brought its cloud service down a while ago.
Amazon in general has done a great job at operations. One manager there told me, "We are experts at dealing with emergancies, because we use Linux." The thing that is more difficult for Amazon, because they lack the systems engineering culture, is to develop complex software systems. So they provide the number 1 cloud service, but they can't offer higher level services like instant e-commerce packages (like MS Dynamic).
Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?
"Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?"
These age old proverbs come to mind:
- Empty barrels make the most noise
- It is better to remain silent and be thought a fool, than to open your mouth and remove all doubt.
I thought Windows Azure is intended to be a sort of proof of concept technology advertisement. A carefully constructed and managed demonstration run by the corporation which creates and sells the underlying software - for the purpose of illustrating to the world what can be expected of it.
It would appear to be fulfilling its role rather well.
Too complicated for even its own super-mega-corp creator to operate? Splendid, I'll have some of that. Where do I sign?
Their Certificate Expiry Tracking program hasn't yet been ported to Windows 8.
yeah, it does have to with competence though or, rather, complete and utter incompetence.
"Eadon sees deeper"
Talking about yourself in the third person?
Thread reply FAIL.
Put it another way: they live in the real world, rather than a world in which what will and won't happen is laid down by the marketing department.
However, yes, this was am administrative affair, not an x-os vs y-os thing. That a company like MS ---isn't it supposed to be entirely up its own backside with procedural stuff?--- could let this happen is ridiculous, but not surprising. About ten or twelve years ago, didn't they forget to renew their own domain? IIRC, some fairly junior employee saved the day, out of hours, by making the payment out of his own pocket. No doubt I have some details wrong: the story will be in the Reg archives.
Procedural and management failure. Really, does it take an MBA to get this sort of stuff right? Don't they have people that actually count beans? Absurd. One of my employers nearly had the power cut off because the ex-sales-side chief thought he had better things to do than to pay the bill --- or pass the job to someone else. How can people be so blind to the fact that the first necessary skill or running a company, of any size, is basic, every-day, bill-paying, floor-sweeping, toilet-cleaning, admin?
On the MS-Software vs Anything-Else front: No it wasn't a software failure, but the recovery process could be a valid comparison. If I was going to put up a penguin icon for this post, and then talk through its backside, I might suggest that, with *nix, it might have been a fifteen-minute job editing a text file. But I would be talking though the penguin's backside. Anyway, I have no clue about the actual problems they faced, so better not talk out of my backside. But hey, who can resist? I never did, and I never would, trust my data to MS software. And, anyway, trusting to "cloud" storage is the silliest idea ever, whatever the architecture.
"It's a certificate issue... Nothing to do with windows or linux"
Well, true, but it says something about Windows admins - best admins administrate UNIX systems ... UNIX admins are better at administrating Windows boxen than "specialized" Windows admins, in my experience anyway.
Can't even keep a certificate up-to-date, shit can happen once - you fire the lot, but twice ???
Your proverbs would sound less trite if they were backed up with counterpoints rather than being presented as the argument themselves.
But, but Eadon is generally right.......
The management of Microsofts only talent is repackaging old software and adding stupid functions to it,
All the software with it's robustness, falls over like a card house in a wind storm.... with ONE little virus...
Oh I mean Microsoft kept Port 80 Open for 15 years after it was identified as a security risk etc., etc., etc...
And they run a 24/7 spy on you network, from their software....
Your already being monitored in real time, all over the internet, from WITHIN Microsoft Widows.
2001 and well before:
Microsoft's Really Hidden Files
Look up this file:
The process known as Media Server Tray Application belongs to software Entriq MediaSphere or MediaSphere by Entriq (irdeto.com).
Description: The file EntriqMediaTray.exe is located in a subfolder of "C:\Program Files". Known file sizes on Windows 7/XP are 360,448 bytes (33% of all occurrences), 368,640 bytes or 372,736 bytes. http://www.file.net/process/entriqmediatray.exe.html
The file is not a Windows system file. The program is not visible. The program is loaded during the Windows boot process (see Registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run). EntriqMediaTray.exe is able to monitor applications. Therefore the technical security rating is 35% dangerous.
In case you experience problems using EntriqMediaTray.exe, you can remove the entire program using Windows Control Panel.
Which feeds data off your windows system to here:
And they do this shit:
Blah blah blah blah................. sun shines out our arses etc. Blah blah blah blah.................
Irdeto Intelligence is the industry-leading solution to identify and track unauthorized digital content across all major Internet protocols including user-generated content (UGC) hosting sites, cyberlockers, peer to peer networks, IRC, Usenet groups and public FTP sites. On average Irdeto Intelligence processes 950 million detections that create over 35 million actionable events each month for its clients.
Irdeto Intelligence tracking services include:
Sample P2P report
Peer-to-Peer (P2P) Monitoring - the industry’s leading P2P platform for monitoring, reporting and enforcing copyright
Scans leading P2P networks, including: Bit Torrent, eDonkey/eMule, Ares and Gnutella to identify individuals who upload client content
Collects identifying information on the first uploaders, tracks propagation and can provide data for evidence packages in the event of possible litigation
Includes tracking by asset, file source, language, user origin and breakouts by unique users and downloads.
Compliant with MPAA file verification standards
Blah blah blah blah.................
Infringement Notices - Irdeto sends more than eight million Takedown Notices monthly on behalf of clients and monitors for compliance, providing reports to copyright holders on who has and who has not complied.
Microsoft's entire history is of spying on all people, through a whole range of methods.
Fuck the Peeping Tom Software Co.
""Eadon sees deeper"
Talking about yourself in the third person?
Thread reply FAIL."
I've seen a few people with runaway egos refer to themselves in the third person. I always wondered why, but it just occurred to me that it's probably how they think people are thinking of them. Or maybe are hoping that it looks more like something that's been said by others, in the hope that it'll become some kind of text-based ear worm. Simple (self-) marketing tactic.
Interesting how he also argues that it's still a WIndows problem. "Sort of." And then goes on to expannd it into everyone who uses Windows is an idiot who doesn't understand anything. He had seemed to be posting more reasonably over the last week, and seems to have regressed since Friday. Gives credence to the jokes about him missing his meds :)
Hmm. Guessing you're a UNIX admin. Probably got your job by spouting the same generalisation to your manager - just what he'd want to hear since the presence of UNIX in-house means he's in the beardy club too ...
MS is fine for the SMB sector. Downtime there isn't measured in dumb ways (employee * revenue per hour * hours) and they just work around any problems.
In a large enterprise, such as a bank, we test properly, rolling system clocks forwards and backwards to see if anything breaks, precisely because we know people forget things such as certificate renewals and the public does not forgive downtime quickly.
My question would be, assuming they used their own certificate server, why wouldn't the server be able to check all the issued certificates and flag any issues, even if there's no code in the cloud software to check things? Not even Ctrl-M to centralise scheduling of tasks? It appears that MS' software works in spite of their procedures rather than because of them. I rather want my cloud management to be all standardised and automated, not built as a one-off.
On the upside, I think we might see so certificate-store date checking appearing in MS software any time now, which is good for customers.
I can't believe people are still used the word "boxen".
I am dealing right now with a domain that has been hacked three times in the space of a week, on a Linux server running Apache. I have to recreate the site because the fine technical folks at this Linux-based host overwrote the backups with the hacker's new site design. Yeah.
I'm currently going through WayBack to get an idea on the site layout (designer lost original files and the webguy who put it together literally went crazy) and found the vulnerability, a known issue with Joomla 1.5 that the host never updated.
Now, do you really want to discuss this? Damn fools and idiots are everywhere and the use all kinds of software -> the discontinuity lies between keyboard and chair, buddy.
Oh more neckbeard FUD from the EadonVerse. Cute but repetetive.
The WORST person for security is one that assumes "my OS is perfect". Because they are blind! That's why the typical Botnet-controller and Trojan distributers are hacked LINUX servers (typically with cheap hosters) and not Windows servers or commercial UNIX boxes. That is why hobby "hackers" deface webpages by the dozend. "Linux is perfect" and so security is taking a back-seat.
Besides: Not renewing a SSL certificate is a major FAIL in management/procedures but at least with Azure it is NOT a security problem. MS actually got THAT part right - no one is getting at the data.
And once you get exposed to the real world you will see stuff that is forgotten a lot. No matter what OS. Like the UNIX guys forgetting to configure a CheckPoint Firewall/One properly for use with ORACLE when a program went from "single developer" to "load testing". The idea was "quick setup now and when we go load testing in three month we have the more complex one done". Nope...
Sure, they changed the SOP after that and all those configs had to be done fully from then on. But the fail happened. And not with a small or generic house but rather a 3000+ employee IT specialist organisation :)
TL;DR. Though scrolling through the message to see just how long it was, the repeated use of 'blah blah blah' was beautifully poetic. (Not ironic, since there's no contradiction there.) Congrats on effectively getting your point across. Still, I'm sure Eadon read it. I'll look forward to having to scroll through all that stuff again, interspersed throughout his future posts ...
I'm amazed they *ever* did. Trying to gain geek cred by showing they've heard 'Vaxen' I suppose.
It's not up to the host to update joomla, it's up to the individual site operator because each joomla installation is local to each site hosted.
You can't expect the hosting provider to go through all their customers, looking for instances of joomla and then manually updating them. Depending on the level of customisation each customer has made, this could cause serious breakage.
Whoever was supposed to manage the site (probably the webguy you mentioned) should have updated joomla, and he failed to do so.
It seemed that Linux did its job if the hackers were only able to deface one site (the one with the hole) and not root the whole server.
As for backups, the host keeps backups to protect against disk failure... It's not their fault that the disks did not contain what they were supposed to, they just backed up what was there. Again the site operator should have backed up their legitimate content.
REM Microsoft business model
REM (c) 1975 by Microsoft Software Co.
10 INPUT OPPORTUNITY$
20 INPUT CLIENT$
30 SET OUTPUT=CLIENT$
40 PRINT "Microsoft can do that"
50 PRINT "We've had a project like that for years"
53 PRINT "We can deliver something better in half the time for half the cost"
60 INPUT BUDGET$
70 INPUT SCHEDULE$
80 SIGN CONTRACT$
90 GET BUDGET$
101 NEW PROJECT$
107 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$
110 PRINT "Microsoft Software Inc is leveraging new technologies for expediteization of revolutionary solutionization"
140 LET BUDGET$ = BUDGET$ + BUDGET$
160 PRINT "Microsoft "; OPPORTUNITY$ ; " is the next generation solution to leverage enterprize monitization"
170 LET SCREENSHOTS$ = RANDOM
180 PRINT SCREENSHOTS$
200 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$
220 LET BUDGET$ = BUDGET$ + BUDGET$
230 GET BUDGET$
240 GET SMALL_COMPANY_EXPERIMENTING_IN_FIELD$
260 LET PROJECT$ = SMALL_COMPANY_EXPERIMENTING_IN_FIELD$
280 RANDOMIZE PROJECT$
290 RANDOMIZE PROJECT$
300 PRINT "Microsoft Software Inc, the worlds leading enterprize solution provider announces "; OPPORTUNITY$ ; " Pro XP 3000"
310 DUMP PROJECT$
320 GOTO 10
"Even if I use the odd tongue-in-cheek stylistic quirks"
What, you mean like when you claimed to have fucked my mum and she asked for more?
I actually wish the moderator had left that one up, so everyone could see what an odious runt you are.
Still we do have standards here. And no, I am never going to let you forget this.
Host and site operator was one in the same - package deal paid quarterly. A 5 man operation that outsourced everything except billing - plenty of them out there. The access Joomla gave did root the server, passwords changed, directories deleted, etc. I have since xferred the domain elsewhere. The host I use and recommend sends email alerts when software used via Fantastico needs an update, has for years.
@Eadon -> Your Momma.
The entire box was hacked, old unpatched Linux and Apache, operated and maintained by MUPPETS.
The majority of Windows users/admins know how to update, the old-school Unix/Linux credo of fire and forget, only checking servers if they catch fire, has been over for some time.
Eadon you are mixing up Security and availability. Security deals with illegal access and unwanted changes to data. Availability deals with being able to access a system. Totally different things. The SSL error means the data is not available. It is neither lost nor compromised. Security is about restricting access to those with the right permissions. Nothing more, nothing less.
As for the rest: Unlike you I understand reality!
Wasn't "The Stylistic Quirks" the name of an early 80s punk/synth fusion group?
Guessing you don't know one programming language from another, let alone what platform you might find it on.
Excuse me while I guffaw, spitting all over my keyboard
As the song says... It all makes work for the working man to do.
But... Linux complacency... Yes, it's stupid. Won't somebody think of the penguins?
There was some lack of detail in the media - the data center itself was not down - my customer has equipment housed there and it was fine. I heard about their cloud having issues - but don't know what they were.
And no, I don't work for IBM...
No, it says nothing whatsoever about that. Stop trying to shoehorn a 'my OS is bigger than yours' argument into everything.
This sucked, but it's a process failure - not a technology or an OS one. It will be interesting to see the RCA and understand how they made such a cock up...
Small bit of pedantry, but Azure storage was actually down for 9 hours rather than the 12 claimed in the article.
>why wouldn't the server be able to check all the issued certificates and flag any issues
A very good point - interesting that in Windows 7 expired certificates only get flagged on screen and are not reported to the event log...
I don't know what Unix/Linux does but I would hope it would get logged and hence picked up by the monitoring system.
"The more they overthink the plumbing, the easier it is to stop up the drain." ... Montgomery Scott
Biting the hand that feeds IT © 1998–2017