Air-gapping SCADA systems won't help you, says man who knows Hoping to keep industrial control systems out of reach of hackers by keeping them air-gapped is a hopeless mission that’s bound for failure, according to a SCADA guru. Isolating SCADA systems as a means of protection has been suggested by some as a defensive tactic after hackers briefly took out elements of the power grid in …
The myth of air-gapped SCADA needs to die once and for all.
On a closed secure site: fine, give it a go. If you can manage to operate efficiently without any link to the outside world then I'm happy for you. Most business don't work that way.
For anything remotely distributed (i.e. most utilities) the air gap WILL be breached somewhere and no, you won't know about it - until it's too late...
"For anything remotely distributed (i.e. most utilities) the air gap WILL be breached somewhere and no, you won't know about it - until it's too late..."
So what can you do about it? You can't go after the face because by the time it's breached, it's already too late, the damage is already done. Yet you're tasked with making sure it's NOT breached for national security reasons. By people who can direct you with legal force, "Stop all breaches. That's an order."
But that's REactive, and as noted, that's not going to work because by the time you react, it's already too late. In a world where a split second is enough, you MUST be PROactive. Yet you're saying you CAN'T be proactive because the only warning signs come AFTER the fact. That logically leads me to believe there's no way to protect a mission-critical system. Meaning we're basically all screwed.
"So what can you do about it? "
The article is about that; it suggests detecting anomalous behaviour on the network to track down breaches.
In the real world the "hacking" will take a long time, weeks or months, so detecting breaches before they can be exploited is the strategy; but it requires investment and an ongoing effort to police the system. This is just like the real world, putting a lock on a door doesn't make it secure; paying guys to keep an eye on things works better but costs more.
Just because you have access to the network doesn't mean you have authorisation to control attached devices; using secure protocols (e.g. HTTPS) and access controls within the network will reduce the ability of an attacker to exploit a breach in access.
To summarise, security costs money; if you cut corners you'll get what you paid for.
"On a closed secure site: fine, give it a go. If you can manage to operate efficiently without any link to the outside world then I'm happy for you. Most business don't work that way."
There is no reason a public utility or any industrial system needs to be linked to the outside world via the public internet. If interconnection to other company sites is required then use a physical private line and make sure its air gapped at the terminus. How to enforce? Have monitor daemons which set off an alarm if any network parameters are changed, have timestamped CCTV in the server room and bring civil and possibly criminal damages against anyone who breaches the air gap without explicit permission from the IT director.
You can't prevent a willful sabateur from removing the air gap but to do so they've got to get into your server room and if they've managed that then you're royally screwed anyway. But you can prevent idiots doing it and saying its not possible is really just an excuse for laziness.
SCADA networks are not connected to the internet, but they are connected to other devices, which are connected to other devices which are etc.
It's all about the layers.
The more layers, the longer it takes to penetrate the system -which (if you are monitoring it properly) gives you some time to implement countermeasures.
One of the issues with identifying weaknesses in the SCADA networks is that any kind of active scan is very likely to break the SCADA device itself :(
it is indeed about the layers. The first layer is, of course, physical security, and should be quite obvious to anyone. The second is the air gap, but the air gap need not prevent access to data from the outside. Think SCADA+DA, where a second data acquisition system, completely independent from the SCADA, is used to connect a separate data acquisition system to the Internet-facing system. The air gap is between the SCADA and the secondary DA. This allows giving data access to the third parties convinced that they somehow need it, while keeping the SCADA isolated and limiting access to the engineers who actually understand it.
Basically, the idea is to limit the number of people who need to access the SC part. When only a handful of engineers need access, it is then possible to use more expensive remote access measures, such as private point-to-point networking.
"There is no reason a public utility or any industrial system needs to be linked to the outside world via the public internet."
Yes there is. Most organisations want to use the data away from the site so it has to be connected and in many cases the end-use is on a device that has Internet access, eg a laptop or an iPad. Many sites are unmanned which is a big cost-saving for the organisation but means they must be connected and allow remote control as well.
Of course you could run in a secure networks, employ security guards, have high fences and guard dogs but unless you're a nuclear facility where the stakes are very high it is unlikely you could justify that. And if the end-use device is on the Internet, which invariably it is, then there is a potential for a breach.
@boltar - "There is no reason a public utility or any industrial system needs to be linked to the outside world via the public internet."
Accounting doesn't need to know in an up to date and accurate manner from production how much was sold at what price (keeping in mind that demand and prices change continuously these days)? Accounting doesn't need e-mail? E-mail doesn't need to connect to the Internet? That's the sort of train of logic you're following. It's the indirect connections through a chain of systems which is where the problem is.
The way that Stuxnet is believed to have got in was through the contract software developer's workstations. The software developer has an internet connection at his office because he needs one to conduct business, as well as to do things such minor things as install (and validate the copy-protection keys for) the eye-wateringly expensive proprietary development software. So, infect the software developer's laptop via a simple phishing attack. The next time the developer goes on site to install updates or debug a SCADA server, the virus is transferred to the SCADA server, or data is transferred the other way (if the virus is already installed). When the software developer returns back to his office and connects to the Internet, the return connection to the virus author is made.
As for a SCADA server, it's just an MS Windows PC, with all the corresponding zero day exploits which go along with that, which can be bought off the black market for the right price. The SCADA server is networked to the actual control system (the PLC normally), and sends it what appear to be legitimate commands and reads back data to display and store in a database (it was an MS SQL Server vulnerability which Stuxnet exploited).
Let's follow the "air gap" logic to its conclusion. By this logic, software developers and system administrators much not ever, ever, at any time have access to the Internet from any system which they have access to at work. Allowing them any Internet access at all for any purpose potentially defeats the air gap. The same goes for accounting, engineering, etc. Internet access must be limited to the top executives and the sales team. Everyone else must work via telephone, interdepartmental mail, and postal mail (with CDs, tapes, and disks being confiscated and destroyed in the mail room). Cell phones are also verboten, and must be left at the door. Simple, right? I bet that someone could still find a hole in this though, like say via contractors or business partners (see above for an example).
What is needed is layered defences to slow down penetrations, plus monitoring. That's how bank vaults work. There is no bank vault in existence which is impenetrable. What there are is bank vaults which take time to penetrate, plus active monitoring which will detect penetration attempts before the vault can be penetrated.
Dont really have much IT experts and just do enough to "make it work" and is effectively prototype software with a glossy UK.
Embedded software on the receiving end is baked in or not designed for tweaking or patching (i.e. replace controls module=software update)
The lower the manufacturing volume the more unique each machine will be with customer or quality driven tweaks making the software less portable and less easy to mass-update. Also machines don't always attract paid maintenance especially when bought second hand and software updates still cost money to produce. Manufacturers close, merge and change hands making continuity tricky too.
There are fundamental issues with the market architecture, not just the software and the issue needs to be looked at from further back to why the current situation exists, not just flap about patching vulnerabilities.
"20 years ago, Faizel Lakhani used a PDP-11 and created the first SCADA system "
maybe that is a misprint for "40 years ago" ?"
That's what I thought too. "Almost infinitely improbable" would be a polite description of the quote in the article.
"SCADA started off with archaic protocols such as FDDI, Token Ring"
That's both incorrect and misleading. FDDI and Token Ring aren't protocols as such. They're (basically) different kinds of cable that allow you to get packets of data from Box A to its neighbour Box B (OSI model is clearly too complicated for somebody in this picture). One could if so inclined run TCP/IP over FDDI, and indeed people have done that (FDDI used to have some interesting high-availability properties now largely ignored). Token Ring? Who cares. Token bus was more relevant (briefly) to industrial control than Token Ring was.
Something like Modbus (or the equivalent from other vendors) is a protocol, and it's archaic, and it's semi-universal in the SCADA world. It can run over many different kinds of cables and connections but typically started life over basic serial comms and evolved in due course to Modbus over TCP or some variants thereof.
Poor article on which to end the week. Could do better.
Have a great weekend.
Disconnect the SCADA system from the network completely.
Connect a PC or use the LCD control panel to display important statuses and alerts.
Aim a CCTV camera at th estatus screen (and if you wan tto go really fancy get it to alert on signifacnt changes in certain areas). Get the remote users to view via CCTV. Any issues they drive to site.
If you need to react quicker then allow a physical connection to be hooked up only once the issue is assessed and only for the minimum period of time necessary (then pull the plug again).
You could feed the status information from SCADA to a networked PC using a serial cable with the RX connector removed at the SCADA so it is send only. The networked PC can read and format the data it receives nicely for a web status page or two and trip the appropriate alerts if something is wrong. Doing that with a CCTV aimed at a monitor seems like a lot of work and error prone to boot...
If someone is able to hack a system over a serial cable with the RX connector removed then they have godlike powers and I bow at their feet. If even reading the SCADA data is a security problem then they still need to protect the networked PC but at least the attack surface is smaller and it is easier to defend a modern PC running a modern and well patched OS than the Windows 2000 PC that SCADA is probably using!
This is quite possibly the most silly article I've seen on El Reg in a long time.
The guy is arguing air gapping doesn't work because a system that isn't air gapped because it has a wifi bridge or whatever isn't secure. Well duh. If you bridge to the outside world then it's not air gapped and whoever linked it is an idiot and should be immediately fired.
Ignoring the fact you're always going to be vulnerable to a spy on the inside doing stuff.....that air gap is still a perfectly valid security measure if you design it properly, keep as little connected as possible down to only where absolutely necessary, and don't allow it to be bridged easily.
Anyone would think he runs a security company that would benefit by people giving up on air gaps and paying his company lots of money to try to secure things other ways.....
"Ignoring the fact you're always going to be vulnerable to a spy on the inside doing stuff.....that air gap is still a perfectly valid security measure if you design it properly, keep as little connected as possible down to only where absolutely necessary, and don't allow it to be bridged easily."
What's he saying is that it's likely a fool's game to try to enforce air gaps because they're too easy to breach. Plus tech is getting so sophisticated they may not even need radio transmissions to work. Worse comes to worse, they can use the interface of the system (that MUST be human-accessible) to bridge the air (maybe even vacuum) gaps.
Hang a sign on the air-gapped system.
WARNING:
This system is high-security and MUST NOT be connected to any wired, wireless or any other form of network. ATTEMPTING TO DO DO WILL BE CONSIDERED GROSS MISCONDUCT AND GROUNDS FOR INSTANT DISMISSAL AND MAY ALSO RESULT IN CRIMINAL PROSECUTION
You may need to change employee's contracts to allow the above to actually be enforceable, but in any security system, the technological solution (firewalls, airgapping, etc) are only part of the solution. Making sure the system's users don't do anything stupid is a part of security as well.
So how do you control a complex power network fully automatically, without connecting SCADA systems to the internet, at least indirectly? This is near impossible, especially considering there will be rapidly varying demand, wildly fluctuating supply from solar panels in various homes, wind turbines on various hills, and a host of power stations needing to adapt their output on the fly, and loads of smart meters trying to get the best deal on that energy market. Do you have people furiously tapping in commands and SCADA control stations all day and all night? However much you wish to isolate critical systems, these critical systems must get data from the real world to control their behaviour. Entering the data in real time can really only be done through some network connection.
I have no easy answer on the security side (security is hard, so pay for it), but a layered approach of some sort seems a likely way to go. I get that Lakhani is a salesman, but that in itself does not mean he is wrong.
Something is either air gapped or connected, there is no half way measure. However, even if properly air gapped, some other means can be found to cause upset, was that apparently genuine e-mail asking for action really asking for the action that someone just reach across to a terminal to type in a commend, or even worse reached under the desk and connected two things together? Security does need to involve people, systems, processes and hardware.
The genie is out of the bottle now so should MMI be properly gated so that commands unsuited to the deployment in question cannot be allowed through however they are entered? That is to say buffer the control system behind one or more intelligent access control/filter points possibly with gating/isolation devices fronting any vital otherwise dumb and witless 'control devices'.
Whenever a statement like this is made: first determine if he's attempting to sell you something. In this case he is. So of course he's going to say anything to get you to look at his solution.
In this case, he's using fear tactics... whenever a salesman does this, run away. If fear has to be used as a tactic, then the product cannot stand on its own or it isn't special or unique.
Second... remember, this is information security... there is no "sure fire, all perfect wall of security".
To say "air gapping" a system is going to fail, because most systems aren't truly air gapped isn't exactly a revelation in line with the burning bush on a mountain. In fact, I'd say it isn't air gapped.
An isolated network (air gapped) used to run SCADA systems is much more secure than a network attached to other networks... which eventually attaches to a cloud of other networks. This is a "duh" moment.
However, no network will be secure unless there are security policies put into place, all devices and systems properly configured, encryption used, monitoring, log management, account/privilege control, etc. You know, the things we call defense in-depth. Just because the system is isolated doesn't mean you can dismiss security devices and defense in-depth. Failure to do so is why isolated SCADA systems are breached.
There are millions of isolated networks running SCADA systems all over the world which haven't been breached. Nearly every large size business uses them. Just ensure you engineer the same security solutions along with monitoring you do with all your other networks enclaves.
Don't let some shady salesman use fear to take your money. You're smarter than this.
"There are millions of isolated networks running SCADA systems all over the world which haven't been breached."
ONLY because they're not worth anyone's time. But what if you're talking a high-profile target, like a nuclear centrifuge, which can draw the attention of agents of the highest sort—from enemy states? You're against an opponent with LOTS of resources, including access to things YOU may not have access. NOW you have to ask yourself: how do you handle THAT kind of opponent without losing yourself in paranoia?
"ONLY because they're not worth anyone's time. But what if you're talking a high-profile target, like a nuclear centrifuge, which can draw the attention of agents of the highest sort—from enemy states?"
I would suggest not buying software from a US based software company if you are somehow running nuclear centrifuges and are not the US.
Be super paranoid.
And move to 1920's technology. Surprisingly hard to hack a building full of typerwriters and pulse dial phones connected to a mechanical exchange.
Also kinda hard to do the necessary calculations, too. Think about the size of the devices used to pull off the Manhattan Project, and that was technology 10-20 years beyond what you're proposing.
Meanwhile, they can still covertly jack the wires. Wasn't Al Capone's South Side Gang phone-tapped during the Prohibition Era?
The design might require a supercomputer.
The SCADA system running it almost certainly doesn't...
Also, physically phone tapping a criminal gang when you're the government of the country they are operating in != phone tapping the government of a country you're a criminal gang in.
But the system regulating the reactor and making the calculations to send TO the SCADA will probably require something on the order of a Colossus if you're using WW2 tech.
And "phone tapping the government of a country you're a criminal gang in." is also != "a state tapping the resources of an enemy state". That's the level of paranoia you need to consider.
"Wasn't Al Capone's South Side Gang phone-tapped during the Prohibition Era?"
Don't know or care, but how many readers understand the significance of the vendor's name in this picture: SS8.
Y'know, kinda like SS7, but with an off by one error. I like off by one errors.
That's great, if the US is your only adversary. You would have to add the UK, Australia, Canada and various other allies to that no-buy list. And insure that all your hardware is securely shipped directly from the factory with no opportunity for someone to intercept it and alter it. And hope that the factory (probably based in China) has good security to prevent it being bugged there. And hope that maybe China's government isn't willing to cooperate behind the scenes on certain things contrary to public statements that make them seem at odds about most things.
I think avoiding US software is only the tip of the iceberg, if you are an Iran or North Korea trying (for example) to set up centrifuges to purify uranium to weapons grade concentrations. You would probably be better off buying some 90s era PCs from a third world school and installing a fresh copy of Windows 98 on them to operate your equipment. Connect them with unmanaged 100 mbit switches. No worries about getting modern malware on them, or the hardware coming prebugged to target you. Go retro, in other words.
No USB ports means no USB sticks carrying Stuxnet. Probably don't even have to worry about boot sector viruses or TSRs anymore, because no one would be left using floppies except you :)
"I think avoiding US software is only the tip of the iceberg, if you are an Iran or North Korea trying (for example) to set up centrifuges to purify uranium to weapons grade concentrations. You would probably be better off buying some 90s era PCs from a third world school and installing a fresh copy of Windows 98 on them to operate your equipment. Connect them with unmanaged 100 mbit switches. No worries about getting modern malware on them, or the hardware coming prebugged to target you. Go retro, in other words."
What makes you think the state adversary doesn't keep retro malware on hand for just such a situation. Odds are they can intercept the computers and infect the BIOS's in such a way they can't be removed.
"No USB ports means no USB sticks carrying Stuxnet. Probably don't even have to worry about boot sector viruses or TSRs anymore, because no one would be left using floppies except you :)"
USB floppy drives still exist, so odds are the state keeps floppy malware as well. Plus, no external drive means no way to transfer the critical (and non-human-readable) data between machines via SneakerNet.
" how do you handle THAT kind of opponent without losing yourself in paranoia?"
The first requirement is to realise that you have that kind of opponent.
Then you design the system to be secure rather than designing the system and trying to bolt security on afterwards.
DESIGN a system to be secure? HAH! That's like wishing for unicorns since NO ONE can take EVERYTHING into consideration when designing a system to be secure. And all it takes is ONE slip for the whole works to be worse than useless. Plus you always have the threat of insiders, which no design known to man can even start to control.
Solve the PEBKAC enigma and the SCADA problem matters not a jot, for Virtual Machinery is ITs own master and favours none but the significantly smarter brave renegade and private pirate partnership..... Absolutely Fabulous Virtual Enterprise with Deep AI into Light and Dark Web Ventures/Black Watch Projects.
Or is that too much like an alien programming for simply complex handling by earthed systems and sysadmins?
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
