The Register® — Biting the hand that feeds IT

Feeds

Microsoft brings Azure back online

Microsoft has managed to repair its Windows Azure cloud, after an expired SSL certificate downed storage and other services for people across the world. Ninety-nine percent of the affected services have been brought back online, Redmond said early in the hours of Saturday morning, Pacific Time. "We will continue monitoring the …

This topic is closed for new posts.

Page:

Paris Hilton

The cloud floats majestically away...

Beijing is going to eat the West for lunch.

Silver badge
Joke

Re: The cloud floats majestically away...

Yes. But because they're using chopsticks it will take a while.

This post has been deleted by its author

Bronze badge
FAIL

20 goto 10

To err once is human.

To err twice takes a super-mega-corp.

Silver badge
Headmaster

Re: 20 goto 10 - MS GOTO FAIL

@ElNumbre - there have been at least 3 - THREE - MAJOR Azure embarassments in the space of a year - there was another one only two weeks ago, that has Microsoft's finest spending a week recovering users' data after Azure crashed.

To be pedantic, only Microsoft fails like this, other megacorps put in place sensible engineering processes.

Amazon and Google and even Salesforece have cloud systems that handle exponentially more data than Microsoft's mickey mouse Azure system. Those megacorps use Linux server systems to do the job.

Bronze badge

Re: 20 goto 10 - MS GOTO FAIL

It's a certificate issue... Nothing to do with windows or linux

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL

This time only with Microsoft, not Windows.

Bronze badge
FAIL

Re: 20 goto 10 - MS GOTO FAIL

This shouldn't surprise us. Their software is absolute shite - what in the world would make us think they were competent to run a chook raffle, let alone an online service?

Anonymous Coward

IBM Too

IBM's "state of the art" Cloud data centre in Auckland was completely down for 30 hours+.

No public expanation.

(after a while the newspaper shut up about how long it was, and just started tell us how many racks it had, and how many backup generators. Edvertorial anyone?)

Re: 20 goto 10 - MS GOTO FAIL

This is a classic example of an operations-management failure. It's got nothing to do with the OS. And if you think only Microsoft has failures, you probably missed hearing about when Amazon's load balancing system brought its cloud service down a while ago.

Amazon in general has done a great job at operations. One manager there told me, "We are experts at dealing with emergancies, because we use Linux." The thing that is more difficult for Amazon, because they lack the systems engineering culture, is to develop complex software systems. So they provide the number 1 cloud service, but they can't offer higher level services like instant e-commerce packages (like MS Dynamic).

Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?

Re: 20 goto 10 - MS GOTO FAIL

"Eadon, if you think Dave Cutler doesn't know how to design an operating system, or if you think Linux never fails without a lot of tweeking and patching, then I gotta wonder why you feel so passionate about a subject you don't actually know much about?"

These age old proverbs come to mind:

- Empty barrels make the most noise

- It is better to remain silent and be thought a fool, than to open your mouth and remove all doubt.

This post has been deleted by its author

Thumb Up

Re: 20 goto 10

I thought Windows Azure is intended to be a sort of proof of concept technology advertisement. A carefully constructed and managed demonstration run by the corporation which creates and sells the underlying software - for the purpose of illustrating to the world what can be expected of it.

It would appear to be fulfilling its role rather well.

Too complicated for even its own super-mega-corp creator to operate? Splendid, I'll have some of that. Where do I sign?

Bronze badge
Pint

Re: 20 goto 10

Their Certificate Expiry Tracking program hasn't yet been ported to Windows 8.

Silver badge
Boffin

Re: 20 goto 10 - MS GOTO FAIL

@Spoddyhalfwit - yes, it's a cert issue - but, you know what? Windows systems seem to be chosen by the types of people that do not understand security very well. And such people are more likely to overlook even basic tasks, like ensuring the PKI certs don't expire.

So it's not a Windows issue, yet somehow it IS a windows issue.

Silver badge
WTF?

Re: 20 goto 10 - MS GOTO FAIL

@Don Mitchell, Actually it is connected with the OS - People that use Windows know or care relatively little about security and therefore it's unsurprising that such people let certs expire. It's not the coincidence it seems, methinks. Eadon sees deeper.

"Amazon are experts at dealing with emergencies because they use Linux?" Sounds like bullshit to me. London Stock Exchange - they switched to Windows and the system crashed. They couldn't bring it up and a Trillion was lost in Trades. They quickly reverted to Linux and have been fine ever since.

FYI Dynamics CRM is Microsofts photocopy of Amazon's AWS and Salesforce CRM. BTW Microsoft were caught astroturfing (again), with an actor posing as a businessman pretending to prefer Dynamics.

"Dave Cutler doesn't know how to design an operating system" - Cutler copied VMS to make the NT kernel. Granted that kernel was OK. But the crap on top of that kernel sucks. The Win 32 API, the bloat that subsequent engineers added to it, the layer-upon layer of crap - apparently there are 50 complex, interwoven layers and a serious engineer can master only about 3 of these.

"if you think Linux never fails without a lot of tweeking and patching," - FUD

"then I gotta wonder why you feel so passionate about a subject you don't actually know much about" - My friend, it seems it is you who is the one who is ignorant of both operating systems and operations. Your entire tirade of clueless PHB bullshit is hereby repudiated.

Bronze badge

@Spoddyhalfwit

yeah, it does have to with competence though or, rather, complete and utter incompetence.

FAIL

Re: 20 goto 10 - MS GOTO FAIL

"Eadon sees deeper"

Talking about yourself in the third person?

Thread reply FAIL.

Silver badge

Re: We are experts at dealing with emergencies, because we use Linux

Put it another way: they live in the real world, rather than a world in which what will and won't happen is laid down by the marketing department.

However, yes, this was am administrative affair, not an x-os vs y-os thing. That a company like MS ---isn't it supposed to be entirely up its own backside with procedural stuff?--- could let this happen is ridiculous, but not surprising. About ten or twelve years ago, didn't they forget to renew their own domain? IIRC, some fairly junior employee saved the day, out of hours, by making the payment out of his own pocket. No doubt I have some details wrong: the story will be in the Reg archives.

Procedural and management failure. Really, does it take an MBA to get this sort of stuff right? Don't they have people that actually count beans? Absurd. One of my employers nearly had the power cut off because the ex-sales-side chief thought he had better things to do than to pay the bill --- or pass the job to someone else. How can people be so blind to the fact that the first necessary skill or running a company, of any size, is basic, every-day, bill-paying, floor-sweeping, toilet-cleaning, admin?

On the MS-Software vs Anything-Else front: No it wasn't a software failure, but the recovery process could be a valid comparison. If I was going to put up a penguin icon for this post, and then talk through its backside, I might suggest that, with *nix, it might have been a fifteen-minute job editing a text file. But I would be talking though the penguin's backside. Anyway, I have no clue about the actual problems they faced, so better not talk out of my backside. But hey, who can resist? I never did, and I never would, trust my data to MS software. And, anyway, trusting to "cloud" storage is the silliest idea ever, whatever the architecture.

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

"It's a certificate issue... Nothing to do with windows or linux"

Well, true, but it says something about Windows admins - best admins administrate UNIX systems ... UNIX admins are better at administrating Windows boxen than "specialized" Windows admins, in my experience anyway.

Can't even keep a certificate up-to-date, shit can happen once - you fire the lot, but twice ???

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Sandtitz

Your proverbs would sound less trite if they were backed up with counterpoints rather than being presented as the argument themselves.

Re: 20 goto 10 - MS GOTO FAIL - Don Mitchell

But, but Eadon is generally right.......

The management of Microsofts only talent is repackaging old software and adding stupid functions to it,

All the software with it's robustness, falls over like a card house in a wind storm.... with ONE little virus...

Oh I mean Microsoft kept Port 80 Open for 15 years after it was identified as a security risk etc., etc., etc...

And they run a 24/7 spy on you network, from their software....

Your already being monitored in real time, all over the internet, from WITHIN Microsoft Widows.

2001 and well before:

Microsoft's Really Hidden Files

http://sillydog.org/mshidden.php

Look up this file:

Windows 7:

EntriqMediaTray.exe

The process known as Media Server Tray Application belongs to software Entriq MediaSphere or MediaSphere by Entriq (irdeto.com).

Description: The file EntriqMediaTray.exe is located in a subfolder of "C:\Program Files". Known file sizes on Windows 7/XP are 360,448 bytes (33% of all occurrences), 368,640 bytes or 372,736 bytes. http://www.file.net/process/entriqmediatray.exe.html

The file is not a Windows system file. The program is not visible. The program is loaded during the Windows boot process (see Registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run). EntriqMediaTray.exe is able to monitor applications. Therefore the technical security rating is 35% dangerous.

In case you experience problems using EntriqMediaTray.exe, you can remove the entire program using Windows Control Panel.

Which feeds data off your windows system to here:

http://irdeto.com/

And they do this shit:

http://irdeto.com/anti-piracy-services/tracking-and-enforcement.html

Blah blah blah blah................. sun shines out our arses etc. Blah blah blah blah.................

Irdeto Intelligence is the industry-leading solution to identify and track unauthorized digital content across all major Internet protocols including user-generated content (UGC) hosting sites, cyberlockers, peer to peer networks, IRC, Usenet groups and public FTP sites. On average Irdeto Intelligence processes 950 million detections that create over 35 million actionable events each month for its clients.

Irdeto Intelligence tracking services include:

P2P chart

Sample P2P report

Peer-to-Peer (P2P) Monitoring - the industry’s leading P2P platform for monitoring, reporting and enforcing copyright

Scans leading P2P networks, including: Bit Torrent, eDonkey/eMule, Ares and Gnutella to identify individuals who upload client content

Collects identifying information on the first uploaders, tracks propagation and can provide data for evidence packages in the event of possible litigation

Includes tracking by asset, file source, language, user origin and breakouts by unique users and downloads.

Compliant with MPAA file verification standards

Blah blah blah blah.................

Infringement Notices - Irdeto sends more than eight million Takedown Notices monthly on behalf of clients and monitors for compliance, providing reports to copyright holders on who has and who has not complied.

Microsoft's entire history is of spying on all people, through a whole range of methods.

Fuck the Peeping Tom Software Co.

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

""Eadon sees deeper"

Talking about yourself in the third person?

Thread reply FAIL."

I've seen a few people with runaway egos refer to themselves in the third person. I always wondered why, but it just occurred to me that it's probably how they think people are thinking of them. Or maybe are hoping that it looks more like something that's been said by others, in the hope that it'll become some kind of text-based ear worm. Simple (self-) marketing tactic.

Interesting how he also argues that it's still a WIndows problem. "Sort of." And then goes on to expannd it into everyone who uses Windows is an idiot who doesn't understand anything. He had seemed to be posting more reasonably over the last week, and seems to have regressed since Friday. Gives credence to the jokes about him missing his meds :)

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @AC 09:15

Hmm. Guessing you're a UNIX admin. Probably got your job by spouting the same generalisation to your manager - just what he'd want to hear since the presence of UNIX in-house means he's in the beardy club too ...

Silver badge
Linux

Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

"I've seen a few people with runaway egos " -

The Wintard AC's (AC because they don't believe - or are ashamed of - their own bullshit) have no technical knowledge so they attempt character assassinations on their technical superiors. Even if I use the odd tongue-in-cheek stylistic quirks - my engineering and operational and market analysis is bang on and you know it.

Unless can contribute, still your tongues, AC's. Listen and learn from a master. (There are those here who know more than me, but they're not AC's.)

Silver badge
Facepalm

Re: 20 goto 10 - MS GOTO FAIL

Meh,

MS is fine for the SMB sector. Downtime there isn't measured in dumb ways (employee * revenue per hour * hours) and they just work around any problems.

In a large enterprise, such as a bank, we test properly, rolling system clocks forwards and backwards to see if anything breaks, precisely because we know people forget things such as certificate renewals and the public does not forgive downtime quickly.

My question would be, assuming they used their own certificate server, why wouldn't the server be able to check all the issued certificates and flag any issues, even if there's no code in the cloud software to check things? Not even Ctrl-M to centralise scheduling of tasks? It appears that MS' software works in spite of their procedures rather than because of them. I rather want my cloud management to be all standardised and automated, not built as a one-off.

On the upside, I think we might see so certificate-store date checking appearing in MS software any time now, which is good for customers.

Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

I can't believe people are still used the word "boxen".

Bronze badge
Facepalm

@Eaton -> Just stop it.

I am dealing right now with a domain that has been hacked three times in the space of a week, on a Linux server running Apache. I have to recreate the site because the fine technical folks at this Linux-based host overwrote the backups with the hacker's new site design. Yeah.

I'm currently going through WayBack to get an idea on the site layout (designer lost original files and the webguy who put it together literally went crazy) and found the vulnerability, a known issue with Joomla 1.5 that the host never updated.

Now, do you really want to discuss this? Damn fools and idiots are everywhere and the use all kinds of software -> the discontinuity lies between keyboard and chair, buddy.

Re: 20 goto 10 - MS GOTO FAIL

Oh more neckbeard FUD from the EadonVerse. Cute but repetetive.

The WORST person for security is one that assumes "my OS is perfect". Because they are blind! That's why the typical Botnet-controller and Trojan distributers are hacked LINUX servers (typically with cheap hosters) and not Windows servers or commercial UNIX boxes. That is why hobby "hackers" deface webpages by the dozend. "Linux is perfect" and so security is taking a back-seat.

Besides: Not renewing a SSL certificate is a major FAIL in management/procedures but at least with Azure it is NOT a security problem. MS actually got THAT part right - no one is getting at the data.

And once you get exposed to the real world you will see stuff that is forgotten a lot. No matter what OS. Like the UNIX guys forgetting to configure a CheckPoint Firewall/One properly for use with ORACLE when a program went from "single developer" to "load testing". The idea was "quick setup now and when we go load testing in three month we have the more complex one done". Nope...

Sure, they changed the SOP after that and all those configs had to be done fully from then on. But the fail happened. And not with a small or generic house but rather a 3000+ employee IT specialist organisation :)

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL - Don Mitchell @Oh4FS

TL;DR. Though scrolling through the message to see just how long it was, the repeated use of 'blah blah blah' was beautifully poetic. (Not ironic, since there's no contradiction there.) Congrats on effectively getting your point across. Still, I'm sure Eadon read it. I'll look forward to having to scroll through all that stuff again, interspersed throughout his future posts ...

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Frankee Llonnygog

I'm amazed they *ever* did. Trying to gain geek cred by showing they've heard 'Vaxen' I suppose.

WTF?

Re: @Eaton -> Just stop it.

It's not up to the host to update joomla, it's up to the individual site operator because each joomla installation is local to each site hosted.

You can't expect the hosting provider to go through all their customers, looking for instances of joomla and then manually updating them. Depending on the level of customisation each customer has made, this could cause serious breakage.

Whoever was supposed to manage the site (probably the webguy you mentioned) should have updated joomla, and he failed to do so.

It seemed that Linux did its job if the hackers were only able to deface one site (the one with the hole) and not root the whole server.

As for backups, the host keeps backups to protect against disk failure... It's not their fault that the disks did not contain what they were supposed to, they just backed up what was there. Again the site operator should have backed up their legitimate content.

Anonymous Coward

Re: 20 goto 10

REM Microsoft business model

REM (c) 1975 by Microsoft Software Co.

10 INPUT OPPORTUNITY$

20 INPUT CLIENT$

30 SET OUTPUT=CLIENT$

40 PRINT "Microsoft can do that"

50 PRINT "We've had a project like that for years"

53 PRINT "We can deliver something better in half the time for half the cost"

60 INPUT BUDGET$

70 INPUT SCHEDULE$

80 SIGN CONTRACT$

90 GET BUDGET$

100 PAUSE

101 NEW PROJECT$

102 PAUSE

107 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$

110 PRINT "Microsoft Software Inc is leveraging new technologies for expediteization of revolutionary solutionization"

140 LET BUDGET$ = BUDGET$ + BUDGET$

150 PAUSE

160 PRINT "Microsoft "; OPPORTUNITY$ ; " is the next generation solution to leverage enterprize monitization"

170 LET SCREENSHOTS$ = RANDOM

180 PRINT SCREENSHOTS$

200 LET SCHEDULE$ = SCHEDULE$ + SCHEDULE$

210 PAUSE

220 LET BUDGET$ = BUDGET$ + BUDGET$

230 GET BUDGET$

240 GET SMALL_COMPANY_EXPERIMENTING_IN_FIELD$

250 PAUSE

260 LET PROJECT$ = SMALL_COMPANY_EXPERIMENTING_IN_FIELD$

270 PAUSE

280 RANDOMIZE PROJECT$

290 RANDOMIZE PROJECT$

300 PRINT "Microsoft Software Inc, the worlds leading enterprize solution provider announces "; OPPORTUNITY$ ; " Pro XP 3000"

310 DUMP PROJECT$

320 GOTO 10

RUN

Silver badge
FAIL

Re: @Eaton -> Just stop it.

@404 - Yes, let's discuss your issue.

You say, "and found the vulnerability, a known issue with Joomla 1.5 that the host never updated."

There you have it! It wasn't Linux that was hacked. It wasn't even Apache that was hacked. It was an old, unpatched Joomla that was hacked. And, given that Joomla was configured with emails and database access, no doubt the hackers were able to access those systems to send spam etc.

Why are you blaming Linux and Apache for dodgy Joomla install?

@404 - you are officially a MUPPET!

Silver badge
WTF?

Re: 20 goto 10 - MS GOTO FAIL

@mmeier - everything I say is correct, it's not FUD. However, let's take a look at your own dodgy comments.

"The WORST person for security is one that assumes "my OS is perfect". Because they are blind!" - Did I ever say an OS was perfect? No I did not. So your argument is the Strawman Attack Fallacy.

Having said that, remote exploits for Linux are rare, but they are quite common for Windows. But the OS is irrelevant for other kinds of attack, such as social engineering attacks.

"Besides: Not renewing a SSL certificate is a major FAIL in management/procedures but at least with Azure it is NOT a security problem"

You don't understand security very well. Security is about letting people ACCESS data (it is also about other things, but permitting access is one of them). If your system does not let people access their data, then your system has FAILED in terms of security.

"And once you get exposed to the real world you will see stuff that is forgotten a lot"

- In the real world people get murdered a lot too. So what? Your argument is ridiculous. I have noticed that in general your comments are spectacularly clueless and this is a typical example. You are one of those guys with just enough knowledge to be dangerous.

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

"Even if I use the odd tongue-in-cheek stylistic quirks"

What, you mean like when you claimed to have fucked my mum and she asked for more?

I actually wish the moderator had left that one up, so everyone could see what an odious runt you are.

Still we do have standards here. And no, I am never going to let you forget this.

Bronze badge

Re: @Eaton -> Just stop it.

Host and site operator was one in the same - package deal paid quarterly. A 5 man operation that outsourced everything except billing - plenty of them out there. The access Joomla gave did root the server, passwords changed, directories deleted, etc. I have since xferred the domain elsewhere. The host I use and recommend sends email alerts when software used via Fantastico needs an update, has for years.

;)

Bronze badge

Re: @Eaton -> Just stop it.

@Eadon -> Your Momma.

The entire box was hacked, old unpatched Linux and Apache, operated and maintained by MUPPETS.

The majority of Windows users/admins know how to update, the old-school Unix/Linux credo of fire and forget, only checking servers if they catch fire, has been over for some time.

This post has been deleted by a moderator

Silver badge
Linux

Re: @Eaton -> Just stop it.

@404 I'm going to give you the benefit of the doubt as I'm in a good mood. Nevertheless your conclusion does not follow from your surmise.

Also - "The majority of Windows users/admins know how to update"

I would aver that I should bloody well hope the majority of windows sys admins know how to update :-)

I very much doubt your average windows user knows how to update, however. Have you met the average windows user? Thick as a bog brush. (Windows Admins are slightly higher up the food chain - some of them have opposable thumbs.

Naturally Linux / UNIX admins are the top predators - the elite.

Re: 20 goto 10 - MS GOTO FAIL

Eadon you are mixing up Security and availability. Security deals with illegal access and unwanted changes to data. Availability deals with being able to access a system. Totally different things. The SSL error means the data is not available. It is neither lost nor compromised. Security is about restricting access to those with the right permissions. Nothing more, nothing less.

As for the rest: Unlike you I understand reality!

Anonymous Coward

Re: 20 goto 10 - MS GOTO FAIL @Phoenix50

Wasn't "The Stylistic Quirks" the name of an early 80s punk/synth fusion group?

Silver badge
FAIL

Re: Guessing you're a UNIX admin

Guessing you don't know one programming language from another, let alone what platform you might find it on.

Silver badge
Coffee/keyboard

Re: In a large enterprise, such as a bank, we test properly

Excuse me while I guffaw, spitting all over my keyboard

Silver badge
Linux

Re: I am dealing right now with a domain that has been hacked three times in the space of a week

As the song says... It all makes work for the working man to do.

But... Linux complacency... Yes, it's stupid. Won't somebody think of the penguins?

Re: IBM Too

There was some lack of detail in the media - the data center itself was not down - my customer has equipment housed there and it was fine. I heard about their cloud having issues - but don't know what they were.

And no, I don't work for IBM...

Bronze badge
FAIL

Re: 20 goto 10 - MS GOTO FAIL @Spoddyhalfwit

No, it says nothing whatsoever about that. Stop trying to shoehorn a 'my OS is bigger than yours' argument into everything.

Mushroom

Re: 20 goto 10 - MS GOTO FAIL

This sucked, but it's a process failure - not a technology or an OS one. It will be interesting to see the RCA and understand how they made such a cock up...

Small bit of pedantry, but Azure storage was actually down for 9 hours rather than the 12 claimed in the article.

Bronze badge

Re: 20 goto 10 - MS GOTO FAIL

>why wouldn't the server be able to check all the issued certificates and flag any issues

A very good point - interesting that in Windows 7 expired certificates only get flagged on screen and are not reported to the event log...

I don't know what Unix/Linux does but I would hope it would get logged and hence picked up by the monitoring system.

Mushroom

Re: 20 goto 10 - MS GOTO FAIL

"The more they overthink the plumbing, the easier it is to stop up the drain." ... Montgomery Scott

Page:

This topic is closed for new posts.