back to article Missed patch caused Equifax data breach

Equifax has revealed that the cause of its massive data breach was a flaw it should have patched weeks before it was attacked. The company has updated its www.equifaxsecurity2017.com/ site with a new “A Progress Update for Consumers” that opens as follows: Equifax has been intensely investigating the scope of the intrusion …

Big Brother

The CIO did know to sell shares, but didn't know to patch

He'd better hire a lawyer. Hope the attorney knows more about the law than this gobbler knows about computers. Don't turkeys strut around?

31
0
Silver badge

Re: The CIO did know to sell shares, but didn't know to patch

The CIO will be fine.

Few million golden parachute and a cushy number someone else.

Feel sorry for the poor buggers at the bottom.

9
0

Re: The CIO did know to sell shares, but didn't know to patch

No club FED for him. Hard to use the 'chute when incarcerated.

0
0
Bronze badge

Re: The CIO did know to sell shares, but didn't know to patch

The CIO will be fine.

Few million golden parachute and a cushy number someone else.

Feel sorry for the poor buggers at the bottom.

Maybe, maybe not.

The way I read @Tree's comment was as referring not to the financial return the CIO made, but to the criminal act that that entails.

If the CIO knew about the breach, then sold the shares before announcing it publicly, that is criminal insider trading. And while rich business people get away with a lot, insider trading is something the SEC tends to clamp down on because it affects other rich people as well. So the CIO (and the 2 other executives who sold shares) could be facing an SEC probe into insider trading which could lead to jail time - not for the lax security that allowed the breach, but for insider trading.

8
0

A by product of this fiasco will be that half of the US will have security checks enabled on their credit accounts.

8
0
Silver badge

Is that a good or bad thing?

0
0
Silver badge

run by Equifax,

0
0
Silver badge

It's a good thing if it's Trump's fault.

1
2
Silver badge

I already put a security freeze on both my and the wife's credit at experian, transunion, and innovis. I couldn't do equifax because shockingly the website refuses to place one but I'm sure it will allow me to sign up for the $19.95/month lock option and the phone system is so overloaded I can't get through even to the "All of our representative is busy" spiel. Seriously, it's been a long time since I heard a busy signal. They do have the option of putting on a freeze by mail but they want a copy of some form of government ID and I definitely don't want them to give them any additional information.

I will be sending them the bill for the credit freezes at the other agencies however.

I'll add that the transunion site has been falling over to this lately also.

1
0
Anonymous Coward

Schadenfreude?

Is it the same thing as 'schadenfreude' if stories like this make me feel so much smarter than those guys?

Hey, poster idea for Despair, Inc.

"There's no 'I' in trusting and naïve.

- - - oh noes, there is!! I'm toast! - - -

5
0
Silver badge

Re: Schadenfreude?

There's no I in "we're all screwed"

11
0
Anonymous Coward

Re: Schadenfreude?

You are right; there's no trusting these idiots.

1
0
Silver badge

Re: Schadenfreude?

You are not a customer, you are the product. So you don't get any say in "trusting"

1
0
G2
Pint

admin/admin

That's amazing! I've got the same combination on my luggage!

:p

12
0
Silver badge

Re: admin/admin

Ah, Spaceballs with President Srkoob lol

7
0
Anonymous Coward

Re: admin/admin

It's ok they've updated it to admin/123456

9
0
Silver badge

Re: admin/admin

Shush, that's my router password.

5
0
Silver badge

Who will watch the watchers?

Apparently not Equifax.

Anyone who still thinks they can trust the bloated kingpin players in banking and finance with their data or their money just isn't paying attention.

10
1
Anonymous Coward

Re: Who will watch the watchers?

- We need to hang a few leading politicians along the way...

1. It'd wake them up as they appear inept and wholly ignorant of everything going on right now from Privacy to Security...

2. It'd compel them to add headcount to Regulators too. Plus, lets offer large bounties for taking down corrupt corps to a wider audience (in sync with new US whistle-blower laws).

- If not, imagine what the next 20 years is going to be like? Read posts by ex-Facebook Antonio-García-Martínez. Pretty sobering!

3
0
Anonymous Coward

'new US whistle-blower laws'

The new laws are having an effect. If they were extended beyond insiders to consumers, it might have a GDPR sobering effect.

Members of the public would be more vigilant or constantly on the hunt for corporate wrongdoing, as they could share in the spoils.

Better than it all going to Central Government anyway, just to be pissed away on some pay-to-play political favor projects etc.

Problem is the 'political will' isn't there. Politicians don't create jobs so they're terrified of corporations closing doors & moving to Laos.

9
0
Silver badge

Re: Who will watch the watchers?

Adding headcount to the regulator won't help. Complying with identity verification (aagh), money laundering regs and sanction enforcement is one of the reasons the banks use credit reference agencies.

We really need a sensible regulator that doesn't apply the same rules to everyone that it does to Kim Jong-un. And a government that isn't a control freak.

5
1
Silver badge
Coat

Typical problem of many large organizations

Dysupgraditis - permanent inability to deploy required upgrades or patches on time, typically caused by fear of breaking infrastructure. Underlying causes: lack of understanding of the existing infrastructure within the organization, lack of infrastructure to perform pre-deployment testing of patches or upgrades, lack of skill to minimize the downtime or risk from the deployment. Known cure: none.

23
0
Anonymous Coward

Re: Typical problem of many large organizations

Hahaha, we do this too :) Lets see what we have...

MySQL 4.0 (Support ended 2008), 4.1, 5.0, 5.1

Solr 1,3,4,5 (Yep. It seems we like installing Solr, but no-one likes upgrading or patching. Latest is 7 btw)

When you tell business types here you want to spend 1 month updating the DB and testing/optimising all the DB queries, after which they have no new features or design to show clients, they tell you where to stick it.

We have a "security" guy, whose purpose appears to be thrown under the bus if we have any security issues to protect the head honcho. He does no security audits, doesn't read code, and isn't interested in tracking used products and CVEs.

7
0
Anonymous Coward

Re: Typical problem of many large organizations

" lack of understanding of the existing infrastructure within the organization, lack of infrastructure to perform pre-deployment testing of patches or upgrades, lack of skill to minimize the downtime or risk from the deployment."

Sometimes there is plenty of skill to minimize downtime, there are testing platforms and there is full understanding of the existing infrastructure.

Sometimes it's knowledge of the existing infrastructure, and knowledge of what the minimum downtime will be and knowledge gained from pre testing that means certain upgrades are delayed. Sometimes suppliers are required to do the upgrade at an engineer cost of £1,500 a day due to their support contracts.

Like useful backups, business continuity, disaster recovery and data/network security - anyone who thinks it is easy probably isn't doing it right (or is running a very small infrastructure).

Nothing to excuse this current Equifax issue which would not have required significant effort, but sweeping statements that it is always incompetence that means every bit of software isn't upgraded immediately after a patch becomes available doesn't understand reality.

Anyone who is so confident put all your external facing systems through https://observatory.mozilla.org and report back your scores for the observatory tests and all the third party tests. I would of course expect you to have 100% or A+ in all of them.

2
0
Silver badge

Re: Typical problem of many large organizations

I do not have a silver bullet, and I do not claim this to be a simple problem. The list of possible underlying causes is, by necessity, short.

The one common factor I found, reading about this and other failures, is the C-level executives oblivious to, or otherwise ignoring, the implied requirements of sane IT infrastructure, and focusing on explicitly stated business needs only.

8
0
Silver badge

Re: Typical problem of many large organizations

It is easy to say in hindsight that this patch should have been applied.

But just look at the volume of vulnerabilities, across all software platforms that a company has to watch and plan patches for.

Even in the most proactive organizations I've come across, planning and testing a patch deployment, and arranging for the necessary reduction in service as patches are rolled out can take weeks or months.

What most people don't take into account is that it is quite frequent that a patch changes a behavior or breaks something. One of the past mantras in changing systems used to be "only change one thing at a time", so that you could isolate which component breaks the system. But with the 24x7 nature of many systems nowadays, service outages are hard to arrange, so patches are bundled into releases. Because you are changing multiple components, it becomes important to test a release before deploying it, otherwise you get panned by customers and the press for not testing before release.

So you're stuck between a rock and a hard place. If you spend time testing, you're open to the vulnerabilities while you are planning and testing. If you shortcut the testing process, then you're open to breaking the services you offer.

In my view, and I think it is a very common view, there are two significant things that have to be done.

One is engineer the systems such that you can deploy patches to subsets of the environment while leaving the service running (for example a leg1-leg2 split), so that you don't have to have as many service outages.

The second is that you split your application up into discrete security zones, with the internet facing systems that are most likely to be hacked only having access to data on a transaction-by-transaction basis, with the data being provided under the control of the next zone in. Although this will not prevent data theft, it will prevent mass data extraction, so long as you have decent monitoring of transaction rates, and intrusion monitoring.

The systems holding the bulk of the data, for example the database servers, are in your most secure zones, and you make sure that even if someone gets into these systems, it is difficult to bulk export data out to the internet.

The more zones you have, the more difficult it becomes to hack in and export, especially if you use different technologies for each zone. Hopefully, with enough zones, one of two things will happen. Either the hacker trips some intrusion monitor before getting too far into the system, or they decide that it is just not worth the effort to get any data.

There are many other steps that need to be taken, but these two will mitigate software flaws, limiting the damage. Unfortunately, they have to be designed in from the beginning, and are difficult or impossible to retro-fit. This means that a small quick-and-dirty proof of concept or pilot often needs to be completely re-designed to make it production ready.

But too often, manglement see a working PoC, and decide that it can just be scaled up, rather than the necessary (and expensive) redesign. To them, it's all extra cost that they can't justify. And because many of the people implementing the PoC, especially if they are using newer technologies, are often younger and less experienced, they're not prepared to push back.

The result? Systems that are easy to get to the data through exploitation of only one or a small number of vulnerabilities, and easy to export the data across the Internet, together with a difficult patching process. A recipe for disaster.

10
0
Silver badge

Re: Typical problem of many large organizations

@Peter Getherhole exactly, you have won "compartmentalization" word in the bullshit bingo. The sad truth is that many C-level executives only know the words, but do not know the meaning.

4
0
Silver badge

Re: Typical problem of many large organizations

This ignores the other factors in this...

Firstly, we have Apache, who reportedly fixed the vulnerability with the release of new code on 7-Mar-2017. What I found interesting was that the Apache announcement for this release contains the following:

This release addresses one potential security vulnerability:

Possible Remote Code Execution when performing file upload based on Jakarta Multipart parser

Namely, no explicit mention of the CVE fixed in the release, making it relatively easy for a busy admin - with several dozen packages status monitor, to downgrade the update from must do to 'pending'. Which leads us to the first challenge: communicating important information to user organisations.

Secondly, we have the other problem of large interconnected IT systems. I suspect that Equifax like many organisations wasn't running the latest releases and were running a few releases behind, so fixing might require doing more than simply running a patch file and thus increase the need for design review, planning, testing etc. which in turn increases the time before the vulnerability is fixed.

4
0

Re: Typical problem of many large organizations

What about a compensating control? You know that a system is vulnerable, and the risk of patching outweighs the [quantitively measured] risk of compromise, but you put in place a compensating control that mitigates the problem temporarily until a patch can be applied. IDS/IPS anyone? Snort SIDs 41818 & 41819 were available from March.

0
0
Anonymous Coward

Re: Typical problem of many large organizations

It's time to name and shame the companies. It's the only thing the C-suite understand.

3
0
Anonymous Coward

Re: Typical problem of many large organizations

> MySQL 4.0 (Support ended 2008), 4.1, 5.0, 5.1 <

CGI - perhaps you've heard of 'em? - is installing software upon one of my clients using an older Apache Tomcat 8.0.12 webserver and they refuse updating it because "it is the only supported and tested version" even if I point to the relevant CVE details. It is a public facing server front end and although isolated, it still needs several ports open to the back end.

Unfortunately the client's CFO is calling the shots and doesn't understand IT, nor respects others' opinions...

4
0
Silver badge

Re: Typical problem of many large organizations

"This release addresses one potential security vulnerability:

Possible Remote Code Execution when performing file upload based on Jakarta Multipart parser"

"Namely, no explicit mention of the CVE fixed in the release, making it relatively easy for a busy admin - with several dozen packages status monitor, to downgrade the update from must do to 'pending'."

Hmm, Remote Code Execution would make any sysadmin's ears prick up. Anything that has a possibility of remote code execution needs to be investigated for risk asap and should get it straight into the "must do" pile.

4
0
Anonymous Coward

Equifax now know as Tepidhaxx

When you tell business types here you want to spend 1 month updating the DB and testing/optimising all the DB queries, after which they have no new features or design to show clients, they tell you where to stick it.

Are you that guy loudly munching a banana every single morning on the southwest corner of the officedrone farm?

0
0
Anonymous Coward

Re: Typical problem of many large organizations

But too often, manglement see a working PoC, and decide that it can just be scaled up, rather than the necessary (and expensive) redesign. To them, it's all extra cost that they can't justify. And because many of the people implementing the PoC, especially if they are using newer technologies, are often younger and less experienced, they're not prepared to push back.

Can't upvote enough.

I have currently a situation where internal customers are asking every day when the system (a banking system of some importance but luckily not Internet-accessible) "will go into production". There is no budget left, the end of "Phase 1" has been reached and the only thing I see is a buggy, unstable mess written in horribad style by too-young consultants who were squeezed hard to "produce something" with no specifications at hand. This is not a complex system, either! A model, state machines, crud operations, and a web interface. Production ready? Uh... come in, close the door.

Luckily security is so tight that it might be impossible to go live. We don't even have a proper local database to do testing, yeah - test data is really important to lock down. One never knows.

2
0
Bronze badge

Heads need to roll.

4
0
WTF?

The information from KrebsonSecurity about how bad their security posture is just amazing. I have yet to understand how none of the professional hackers haven't plundered the other sites by Equifax.

I mean, seriously, admin/admin and employee password is the same as their user IDs? Really?

I think the company's CIO/CTO needs to be taken out the back and hanged (before being shot).

This is like saying that Equifax has a door but don't have the walls and roof. I hope the SEC will throw the book at `em.

5
0
Silver badge
Joke

RE: Equifax's CTO/CIO

<quote>I think the company's CIO/CTO needs to be taken out the back and hanged by the balls (before being shot.</quote>

FTFY!!!

3
0
Silver badge

Consider the brave new world of microservices deployed as blobs onto docker containers.

A single developer can generate a handful of these a day, possibly in multiple languages with multiple libraries. So over time there will easily be hundreds floating around, nobody really knowing what they do.

Keeping track of those may, just about, be possible because it can be automated. Patching and testing will rapidly become infeasible though. It's taken my team weeks to upgrade five services to the new version of Dropwizard released on Aug 24th (because of all the linked dependencies).

3
0
Anonymous Coward

"leading, independent cybersecurity firm"

Who could this be? The other vendors got owned by FireEye if memory serves me right.

0
0
Silver badge
Facepalm

Well, I suppose it's more than likely now that Equifax's UK customers will also be affected.

Joy.

3
0
Anonymous Coward

Customers? Err no... I'm not a customer but it's probable that my details have been passed to them by banks, utility companies etc that I am a customer of...

Parasites such as Equifax, Experian etc need shutting down ASAP. They're quickly becoming the problem, not the solution.

8
0
Silver badge

Obligatory commitstrip : http://www.commitstrip.com/en/2016/10/14/good-old-adminpassword

3
0

running with admin/admin tells you all you need about how much that joke of a company takes data security seriously.

they deserve to be hung drawn and quartered.

2
0

Doing the bare minimum

Q.E.D

(Latin)

2
0
Bronze badge

"Equifax had nine working weeks in which to apply the patch."

So that they finger pointing has just made you look stupid Equifax.

"The company also appears to have suffered another data breach, this time in Argentina where its Bryan Krebs reports “an online portal designed to let Equifax employees in Argentina manage credit report disputes from consumers in that country was wide open, protected by perhaps the most easy-to-guess password combination ever: “admin/admin.”

You would have thought that after a massive security breach they would be checking everything they have including their underwear. More lessons are going to be learnt at Equifax I think.

Admin / admin you couldn't make this up.

1
0
Silver badge

More lessons are going to be learnt at Equifax I think.

"More lessons need to be learnt at Equifax I think" would be closer to the truth. There are no grounds for believing that the lessons are just "going" to be learnt as if by osmosis.

6
0
Anonymous Coward

Speed kills

It's time to finally admit that the agile, CI, devops, etc. hype train comes at a cost. When the only metric that matters to the business is how fast you can make changes, you end up here. It's time to slow down and address that giant elephant in the room, (Although I know we won't. Even when the elephant's shitting all over your carpet.)

8
1
Silver badge
Facepalm

Re: Speed kills

You are very wrong. This has nothing to do with agile, CI or devops, which are all Good Things development-wise because most developers can't design or find their own arse in the code they produce (I have seen nominal "experts" at work ... hahahah!).

No, this is a question of allocating enough horsepower to maintenance, patrolling and defense-in-depth.

Its' the delusion of the "the system is stable and in production, we won't touch it, DISPAND THE TEAM". Which is is the anithesis of agile, CI or devops.

3
4
Anonymous Coward

Cashless or clueless?

We all know how easily bank and commercial systems can be penetrated and frauds committed. It seems the convenience factor of the cashless society is for the benefit of the fraudsters. With this data I could be left literally cashless.

Lets have a cash only day to show our discontent every month. The retailers will soon complain to the banks when they lose sales after you refuse to pay by any other method and explain why. This includes refusing to use any form of online shopping. That would be truly disruptive.

Does anyone have a suitable date to propose?

2
0
Silver badge
Megaphone

Re: Cashless or clueless?

<quote>Does anyone have a suitable date to propose?</quote>

In the USpfA, I propose """Black Friday""" (for obvious reasons).

1
0

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Forums

Biting the hand that feeds IT © 1998–2017