Feeds

back to article Water utility hackers destroy pump, expert says

Hackers destroyed a pump used by a US water utility after gaining unauthorized access to the industrial control system it used to operate its machinery, a computer security expert said. Joe Weiss, a managing partner for Applied Control Solutions, said the breach was most likely performed after the attackers hacked into the …

COMMENTS

This topic is closed for new posts.

Page:

Alert

Optional

YTF was a water company's control system attached to the net?

12
0
Silver badge

Exactly. It is not the job of the US Government to protect this stuff it is the job of the utilities to not connecting to the rest of the World so we can all have a crack at it. Seriously, these SCADA systems are connected to the internet through shear fucking laziness so these wankers don't need to get off of their fat arses in their comfy offices in order to tweak shit. Well that's a mighty high price to pay if you ask me.

7
2

Not lazy. Cheap.

At the very least, a manually operated system reuqires a nearby "on call" operator. More likely a permannent on site operator, who would spend most of his time with his heels up waiting for something to happen.

Truly dedicated communications infrastructure is also prohibitively expensive.

The true problem is piggybacking something as bloody simple as industrial control systems on top of any complex operating system. Water pumps just plain don't need to be able to run a word processor, a database and 50 active porn windows.

7
0
Silver badge
FAIL

Yep, the real issue is about cost-cutting. They could make this thing secure* by having a dedicated WAN or a VPN instead of using the open web. That would cost a lot more, so they cut corners.

*or as close to secure as is possible

2
0
Joke

*snickers*

" Water pumps just plain don't need to be able to run a word processor, a database and 50 active porn windows."

Sure they do. So the operator can draft his resum^H^H^H^H^H CV as outlined by some twat named Dominic Conner from The Register, the database to index the pirated music and movies because you can never be too organized and the porn windows for the down time.....

1
0
Mushroom

When does the government need to get involved?

So far, we've had hackers able to disrupt Nuclear power stations, open death row and now destroy water pumps.

In short, everything a foreign power would need to do to ensure that the country was, in effect, disabled.

Before you say, that the armed forces would still be available, they should be. However, I suspect they'd be busy either distributing the remaining supplies to the population, as well as attempting to control uprisings.

0
0
Anonymous Coward

wait till we get smart meters!

3
0
Silver badge

It's not up to the government to protect or fix it,

but under the umbrella of ensuring homeland security, it is a responsibility of the DHS to notify critical infrastructure stakeholders whenever and wherever a system vulnerability has been discovered.

I'm not discounting the lazy/cheap factor for the utilities, I just don't want DHS's lazy/cheap factor disregarded either. Particularly since the DHS statement about not being aware of any such vulnerability further enables the lazy/cheap utility.

0
0

Still too effing complex. Too stupid to be hacked...

...is the only way to go.

1. Hardwired (not coded) refusal to exceed "normal operational parameters" by more than x%.

2. A very limited command set.

3. Chained encryption of commands and responses with a null operation failure mode, and a hardwired restart sequence.

1
0
Vic
Silver badge

> VPN instead of using the open web. That would cost a lot more

A VPN would cost *pennies*

They really are very cheap. But you need a PHB who will actually *listen* to such proposals, rather than just ignoring everything he doesn't understand...

Vic.

3
0

Yes. Public infrastructure in the US is almost by definition old, decaying, underfunded, understaffed and neglected, or outsourced to private companies where all of the above applies. One obvious symptom is how the American electrical grid from time to time suffers enormous blackouts that should be easily prevented. The real surprise to me is that things like these don't happen more often.

1
0
Anonymous Coward

Firewall? Guys!

So, these guys that have water pumps connected to the Internet doesn't know what a firewall is?

Surely anyone worth their salt would have systems screwed down - with a basic IP access control list if nothing else - to accept connections only from other authorised IP ranges elsewhere within the organisation?

Doh!

Grendel

7
0
FAIL

But a firewall is still a vulnerability....

It's a water pump for a water system. The water pump and everything it needs to be connected to should be on a completely separate, unconnected to the internet, network.

3
0
Bronze badge

Whatever happened to...

Good old fashioned point-to-point modems? Perhaps with a secured PC on the end to SSH onto the controller. No new-fangled Internet to worry about.

I doubt bandwidth is needed in this application - its a pump - not exactly streaming HD porn....

0
0
M7S
Bronze badge

@ Martin, see Black Betty's comment above

I've a small unmanned pumping station near me, but over a mile from any other significant comms infrastructure. I suspect its not the only one like this and I'm not far outside the M25. I would expect distances in the USA to be even greater so connecting to the "public" net is the only viable option, not that this would excuse any other security failings, but I dont have enough information on that to feel able to comment reasonably.

0
0
Silver badge

Firewall is not the whole solution

The chances are that some malware found its way onto a PC used by one of the maintainance engineers and the attack was launched from there. This would have been deemed 'safe' by any firewall.

This is as much a MS Windows problem as a SCADA problem.

4
0
Silver badge

Grendel, Martin, Hello? Anybody Home?!

Stuxnet was designed specifically to jump the air gap for the ideal system you describe. So Firewalls and air gaps are irrelevant. And the new malware is just a variant using a different hole.

1
0
JGT
Alert

You'd be surprised

Over a decade and a half ago I took a short flight in the LA basin and sat next to a water district employee. We ended up talking computers communications and he mentioned that a city's water system used microwave links for command and control. I asked what kind of encryption they used. He was confused, wanting to know why they should be encrypting the link, it was only doing water system stuff. I mentioned how it wouldn't be hard to override the HQ signal with a correctly oriented and stronger signal. He asked why they should be concerned. As this was before 9/11, I struggled to find a convincing reason and came up with, what if they override the pump controls and over pressurize the system?

He was shocked at the idea.

Seems I wasn't too far off the mark.

0
0
Anonymous Coward

So unnamed ebil actors from far, far away gone and done did...

... destroyed the thing by turning it off and on again?

Some IT "expert", this.

Then again this is the land of dumping a lake worth of drinking water because one drunk yoof peed in it. Yet they couldn't be arsed to put the simplest of fences around it. It was also right open to the elements. Acid rain and all that. You know, acid? And what about the birds? Anybody think of that, huh? And the fish? They fscking fsck in it! And... oh dear I just realised yoof will have been drunk on American Beer[tm]. I can see they feel there's a problem right there.

3
5
Joke

American Beer

Of course they could have just sold the water direct to the breweries and they could have bottled it without further processing.

4
0

You forgot the homeopathic problem.

2 million fish making their messes in a reservoir? no problem, ditto with flocks of 'bird flu' victims crapping all over it.

One person's pee? I'm surprised there wasn't some sort of homeopathic event horizon thing going on..

ttfn

0
0
Ru
Trollface

One word for you:

Homeopathy.

Dilution a billionfold will not save you from the wee.

2
0
Ru
Unhappy

@Paul_Murphy

Ach, beat me to it. Cursed moderation delays.

0
0
Gold badge

Some IT expert...?

Yes you can destroy a pump by turning it off and on again. I'm sure he's capable of talking to a pump expert, and finding out these things.

I can think of many easy ways to break a pump from a remote controller.

Pump motors are designed not to be started too often. Usually they're set up not to be able to start more than 12 times an hour. This is either enforced in software, or often a run-on timer that runs the motor for a set time. Thus, after a start, the pump will idle until it's needed again, or the timer runs out. Bypassing these controls, and constantly re-starting the motors would break them.

If you could get sufficient control of a water supplier's systems you could cause some interesting explosions in the pipework. Water is heavy, and under tens of atmospheres of pressure you can do some serious damage with it. They're possibly lucky they're only having to replace a burnt-out pump motor.

3
0
Anonymous Coward

"Yes you can destroy a pump by turning it off and on again."

Sure. But I was talking about IT "experts", not pump experts. And that makes it (a poorly executed, I freely admit, but still) a joke of sorts.

On a more serious point, there probably ought to have been interlocks preventing software from destroying hardware, and if the lesson is that this requires hardware interlocks that you can't override in software, well, then maybe we should require exactly that. In law if we have to.

This, or even stuxnet, isn't quite the first time software deliberately and spectacularly broke hardware. Is it really too much to ask of experts in this field to know this and learn from lessons past?

0
0
Anonymous Coward

re homeopathic dilution

the magick ingredients that the homeopaths use, do they rapidly breed and multiply like some of the organic stuff in the snow-yellowing fluid?

just asking, like....

0
0
Anonymous Coward

No

Homeopathic ingredients grow by getting fewer and fewer.

Anyway, the moral of this story is that computers can automate everything, including attacks.

0
0
Silver badge
FAIL

"What the hell is going on with DHS? Why aren't people being notified?"

Someone is being stupid here. And it isn't fat-assed bureaucrats safely encysted in the inner belly of an *enormous* dinosaur, blowing tens of billions and busy seizing spanish soccer websites (diversification into IP problems is always good), setting drones on Mexicans trying to cross the southern rabbit fence, using government credit cards to build a home brewery, checking out the 99% or warning about the mighty danger of poisoned buffets.

Did I mention that DHS was involved in blowing a whole New Orleans?

Really, we need a FAIL + NUKE + TRASH icon, all mashed together into a single gigantic clusterfuck. Probably a TAX icon, too.

0
2
Thumb Up

You Sir, are a very cunning linguist.

0
0
Silver badge
FAIL

Idiots

"raised serious concerns about the ability of the US government to secure critical infrastructure"

It is not there job to do so, it is the water company.

But maybe if the US gov made the CEO & MD of such corporations liable for gaol time for allowing such a serious breach of good practice, i.e. putting critical infrastructure on the 'net WITHOUT the software suppliers (MS et al) backing that up with a matching warranty of fitness for purpose, might just help to get such things fixed though.

5
1
FAIL

Toilet pwnage!

"But maybe if the US gov made the CEO & MD of such corporations liable for gaol time for allowing such a serious breach of good practice, i.e. putting critical infrastructure on the 'net WITHOUT the software suppliers (MS et al) backing that up with a matching warranty of fitness for purpose, might just help to get such things fixed though."

How does a government or a court imprison itself?

1
0
Silver badge
Devil

Not quite

You will find that in order to operate bits of critical national infrastructure like water, sewerage, leccy, gas, etc you need to do comply with some reqs. So in fact, the CEO and MD are liable for at least something as they are in breach of their regulatory regime. Similarly, even in the USA the government has quite enough leverage to make such companies do things.

In any case, this just goes to confirm something I have been saying for ages - SCADA security is sh*t. The scariest bit is that the same companies and people who write scada now write smart metering software. So a system with the same lousy level of security as the one on that pump (or worse) will be in every house in a few years in control of leccy, gas and water.

2
0
Anonymous Coward

Meaningful fines would go a long way, perhaps something like 10% to 50% of sales for the year -- or more.

0
0
Silver badge
Facepalm

How does a government or a court imprison itself?

The same way they always do.

Set up a commission of enquiry, find that they have been a naughty boy, and promise faithfully never to do it again. Ever.

Then award themselves more taxpayers money to expand their empire.......

I wish that, when a shortcoming is found in government work, people wouldn't be quite so free with calls for increased money to be spent in that area. It's a recipe for shortcomings to be found everywhere.....

0
0
Silver badge

@Toilet pwnage!

I had assumed that most US utilities were private companies doing the gov work. Even so, you find those who made the decisions and work up to the top, as you can still gaol government or court employees:

Why was it on the net? Ah, probably to save money.

Were the risks considered? Probably not, or ignored to save money.

Who ultimately took the decision (or applied budgetary pressure) that traded-off safety for running cost, and was that an acceptable risk or one that represents criminal negligence? If is was a windows-based box with hard-coded passwords, then negligence is the only answer.

1
0
Anonymous Coward

But maybe if the US gov made the CEO & MD of such corporations liable for gaol time for allowing such a serious breach of good practice, i.e. putting critical infrastructure on the 'net WITHOUT the software suppliers (MS et al) backing that up with a matching warranty of fitness for purpose, might just help to get such things fixed though.

Can we just get real.How serious was this? Everyone makes mistakes, anyone who works in software or IT shoudl be very conscious of that. The level of checking, redundancy and analysis of a system should depend on its criticality and impact on safety. If everything has to be checked to the level that a safety related system has to be checked everything would grind to a halt. There is no indication that this pump was a safety critical device. If every time any employee in a company did something which was not good practice whether the system concerned was important or not then every CEO and MD would be in jail. There already (at least in the UK) neligence laws about failing to take precasutions when dealing with safety related systems.Company directors are personally responsible in these cases.

It may even be that at the current time putting such a system on the net without VPNs etc wasthe correct decision. If the system is non-critical, and if there is a substantial saving in time and effort from remote servicing, and if the additional cost of securing it outweighs the risk of an attack, then actually the appropriate things is to connect it to the internet. The problem with risk is we never know for certain.This is deliberate critminal damage to the pumop and it is very difficult to see any benefit to the perpetrator so why the F**** did they do it. WIth all the publicity I am sure that the risk benefits are now heavily skewed so that all similar devices for that company must be secured.

0
2
Silver badge

@AC 12:16

"Can we just get real.How serious was this? Everyone makes mistakes, anyone who works in software or IT should be very conscious of that."

You are right to a point, in this case no serious damage was caused to the population, etc. However, we are in 2011 and the vulnerability of computer systems, in particular anything using Windows, has been amply demonstrated for all of the last decade.

What this incident shows is a system that might have been fine off-line, without a half billion PCs potentially able to probe it, but clearly was not good enough. With a bit more effort & synchronisation perhaps a determined perpetrator could have wreaked havoc on most of the pups in a region, leading to the possibility of death or injury from disease or dehydration caused by a failure of such a fundamental human need: fresh water.

My point comes down to poking those in charge with a big pointy legal stick (not unnecessary prosecution of genuine mistakes) so that changes are made, and stupidly vulnerable systems (think Siemens and their SCADA's hard-coded passwords) are kept well detached from the internet in the future.

"This is deliberate critminal damage to the pumop and it is very difficult to see any benefit to the perpetrator so why the F**** did they do it."

Two possible answers spring to mind:

1) There is no reason. Just done for idle amusement.

2) Practice for a cyber-attack or a blackmail attempt.

1
0
Silver badge
Facepalm

Doh

Seems we both struggle with 'pumps'

0
0
Silver badge
Flame

There is NO SUCH THING AS A MEANINGFUL FINE on a PUBLIC Utility.

The government guarantees a certain level of profitability for them to operate the utility, and thus the costs of the fines always gets passed to the consumer.

Idijits!

2
0
Silver badge

Sorry, it really isn't difficult for me

to imagine this being leveraged to a major incident with catastrophic consequences. I just don't see need to put fuel on the fire by publishing it.

0
0
Silver badge

Hmmm...

"If every time any employee in a company did something which was not good practice whether the system concerned was important or not then every CEO and MD would be in jail."

You say that like it would be a bad thing.

1
0
Thumb Up

@Paul Crawford

3) Proof of ability - You can see we can do it, just deposit $1million in this numbered account along with the name of your target.....

0
0
FAIL

duh.

Even people with tiny recording studios know not to put their critical machines on the net.

3
0
Devil

Yes, but...

"Even people with tiny recording studios know not to put their critical machines on the net."

Yes. WE know that.

Unfortunately the 1% 36sqm walk-in-robe owning senior management haven't got a clue beyond what wine they will drink for dinner tonight. They expect the untermenschen lot will take care of this, and correspondingly cut the middle-mungers budgets, who then immediately outsource to China, India or Smellistanumboto.

Then everyone wonders why their city water supply is run by a PIC16F84A with a USB interface nailed to a Nokia 8210.

9
0

"Then everyone wonders why their city water supply is run by a PIC16F84A with a USB interface nailed to a Nokia 8210."

I suspect that such an arrangement would be more secure than some of the real solutions out there.

5
0
Anonymous Coward

ya know, that 'F84/8210 mashup is probably more secure....

0
0
Anonymous Coward

Siemens again?

Or someone else?

0
0
Anonymous Coward

Hindsight is 20 20

It's easy to criticise the people who set this up; but the reality is that these systems started out as purely internal networks of RS485 connections, with external access for engineers provided by modems. These have grown in complexity over decades, and probably got connected to the internet long before it became apparent that this would be a threat of any kind.

You can't just insist that systems aren't accessible from the Internet either; there are good reasons, in a country the size of the USA, to ensure that experts can access systems remotely. The challenge is to allow remote access only to authorised persons.

Like a lot of IT specialities, security is coming to the SCADA game a long time after it started; experts in SCADA aren't necessarily experts in security, it is IT management's job to link the two groups together to solve the problem.

0
1
Anonymous Coward

I just don't buy that argument

It's been pointed out for years that nobody in charge put half a thought to securing their systems yet already had them hooked to various private networks, dialin modems (wardialing, anyone?), whatever, and then proceeded to hook them up to the wider internet. Meaning that they did have externally accessible systems and just didn't stop and think what making them more and much easier accessible would mean.

That is deliberately ignoring the consequences of your actions.

And for critical infrastructure that's pretty much inexcusable. We knew the internet is not a safe place; that's been painfully apparent for decades now. Saying "we didn't know" should count as criminal negligence. Having been alerted years ago to possible impending doom but wilfully ignoring it, wondering why the shit hits the fan now is not hindsight at all. It's being too bloody late reading the signs on the wall.

2
0
Anonymous Coward

Bwahaha

@Wombling_Free You owe me a new keyboard.

Memo to self, NEVER drink coffee, especially hot coffee when reading El Reg, I had hot coffee coming out of my nose! OW!

BTW is it true that Nokia 3310's are really used to control stuff? I always wondered why the price on Greedbay seems to be so high.

-AC/DC 6EQUJ5

0
0

Page:

This topic is closed for new posts.