back to article UK ICO, USCourts.gov... Thousands of websites hijacked by hidden crypto-mining code after popular plugin pwned

Thousands of websites around the world – from the UK's NHS and ICO to the US government's court system – were today secretly mining crypto-coins on netizens' web browsers for miscreants unknown. The affected sites all use a fairly popular plugin called Browsealoud, made by Brit biz Texthelp, which reads out webpages for blind …

Silver badge

Another good demonstration of why ad blockers and script blockers are essential.

I see the ICO site is down for maintenance at the moment, I guess someone's pulled the plug on it until they can fix it properly.

43
3
Anonymous Coward

"Another good demonstration of why ad blockers and script blockers are essential."

And also of why dynamically loading scripts instead of using a local copy is a bad idea. You end up with multiple dependencies completely outside of your control.

57
4
Bronze badge

"Another good demonstration of why ad blockers and script blockers are essential."

We...ell... there are people who have poor (or no) eyesight. My guess is that they would tend to trust that specific plugin - or at least grudgingly accept it (though I believe you would maybe best run a dedicated text to speech engine locally, but many websites are just sh*t when it comes to being accessible - just try to open them in lynx... my guess is if it does not work there you are out of luck). Maybe most of us who do run NOscript (or so) do have some exceptions that are more or less mandated by the sites we use? Or that make a website usable? Sure, I allow only a few exceptions, and only temporary, but that's all it would take.

So: yes, script blockers are needed and very sensible, but in some cases we poke the holes into that layer of security - out of necessity. And don't give me the "then don't use those websites if they are unusable without scripts"-thing. Some sites mentioned are university homepages. Imagine being a part of that university and you have to organise your work, teaching, learning, outreach, travel claims, .... through their system, which does pull in such a script. In the present case you were lucky if you are sighted and thus could just refuse to run that script - next time it is a script that is actually needed for the operation of the website.

36
0
Silver badge

@AC

And also of why dynamically loading scripts instead of using a local copy is a bad idea. You end up with multiple dependencies completely outside of your control.

True, but on the other hand, massive amount of work for you making sure your local copies are always up-to-date, and have the latest fixes applied for that nasty little zero day.

We can't really win!

8
7
Silver badge

"True, but on the other hand, massive amount of work for you making sure your local copies are always up-to-date, and have the latest fixes applied for that nasty little zero day."

But when you discount all the non-essential stuff like multiple analytics scripts and calls to left-pad scripts you could write yourself and typefaces that have just that special version of the letter "i" your designer wanted and all the "social media" scripts you link too to show their icons, just how many are left to track and update?

27
3
Silver badge

"True, but on the other hand, massive amount of work for you making sure your local copies are always up-to-date"

In this case it was the "latest version" that was the problem. You don't need the latest version, you need a good one.

25
0
Silver badge

The two problems with hosting* a local copy of dependencies.

- You don't benefit from the browser cached version that almost certainly exists due to the user previously visiting a site with that plugin.

- You have to pay for that bandwidth**

At the end of the day, you have a trade-off because you need to decide whether to trust a third party to manage risk on your site or whether you want to vet everything. SRI would have prevented this specific hack, true, assuming though you didn't have website authors who saw the error in the console, got the new hash, then blindly applied it to put out the "our customers can't login" fire. It also means that where there is a legitimate vulnerability in the framework, your site cannot be fixed automatically.

*Making a copy of any version that you deploy is important if only to deal with one of these vendors disappearing without notice.

**The irony isn't lost on my me.

8
1
Boffin

Content-security-policy

Modern browsers all support content-security-policy, an HTML header which allows websites to white list JavaScript sources. But that would require them to *actually* know where their JavaScript comes from. That would totally break their shitty ad model.

16
1
Anonymous Coward

Cant win...

Firms get together, chip in for a common checking authority, and get audited software.

1
0
Silver badge

Re: Cant win...

Use a local proxy to cache all your remotely collected scripts. Have that proxy run a comparison check against the last known good version for all external scripts.

If the code changes, don't update the cache until it has been signed off as safe, at that point you can update your 'known good' version and carry on serving it to your clients.

Ok, so if there's a problem with a valid script and it needs to be updated then that fix might be delayed until you can sign off the update, but that's a lot better than taking the chance of feeding your customers compromised scripts.

This avoids the need to micro-manage all the scripts internally, but injects a safeguard against compromised remote script updates such as the one in this story.

Or does that sound too hard?

4
0
Silver badge

"We...ell... there are people who have poor (or no) eyesight. My guess is that they would tend to trust that specific plugin - or at least grudgingly accept it"

True. But that doesn't mean everyone else also has to trust that specific plugin and have it running in the background every time they visit a site. You're correct that perfect security isn't possible and that at some point we have to trust something, but there's a huge difference between allowing some exceptions where necessary, and expecting every user to blindly load hundreds of external resources just in case a tiny proportion of people might need them at some point.

That's why script and ad blockers are essential. Yes, you need to make some exceptions and so can't eliminate all possible vulnerabilities. But a big problem with the internet in its current state is that the default expectation is that everyone will always trust and run everything regardless of whether there is any reason to do so. Until the opposite is true and the default is to access resources only when actually needed, blockers are necessary to enforce that state.

8
0
Anonymous Coward

> And also of why dynamically loading scripts instead of using a local copy is a bad idea.

Very categorical that assertion. In reality there are advantages and disadvantages to each approach, as is typical of engineering problems.

3
0
Anonymous Coward

> And also of why dynamically loading scripts instead of using a local copy is a bad idea. You end up with multiple dependencies completely outside of your control.

From the link in the article:

/* [Warning] Do not copy or self host this file, you will not be supported */

0
0
Anonymous Coward

> True, but on the other hand, massive amount of work for you making sure your local copies are always up-to-date, and have the latest fixes applied for that nasty little zero day.

Not to mention the underlying assumption that your own security is going to be any better than the original host's or relevant CDN.

3
0
Anonymous Coward

> But when you discount all the non-essential stuff

Would you describe a screen reader as "non-essential stuff"?

3
1
Anonymous Coward

Re: Cant win...

> Or does that sound too hard?

Is that what you actually do?

0
0
JLV
Silver badge

Yes, but if you take the very useful NoScript, a typical website calls on 8-10 js domains.

It's not always clear what scripts are need-to-have vs eye candy/ad serving. Then, as you enable some, they in turn want to load more.

Obfuscated urls may look malicious but often are just how CDNs work.

(not to mention js-only sites that render nothing until enabled)

It's a massive obnoxious mess and often results in me leaving a site on arrival.

In addition to checking vs last-seen versions, as suggested here, maybe it's time browsers/plugins look at behavior and profile of scripts, including stuff like api calls being made, cpu use and clipboard access. Known-good script hash checks vs a central registry. Possibly uploading never seen code up for analysis.

No need for the equivalent of flashlight apps accessing contacts, for example.

I don't see this as a JS fail per se - any language holding its role in our current web architecture would cause this. But our trust model feels a lot like when we were sending each other EXEs with funny contents in the mid 90s. I don't see it lasting another 20 years.

5
0
Silver badge

Re: Cant win...

"Is that what you actually do?"

Me? No, but it would be something I would look at doing to see if it's workable. Just applying common sense and basic principles.

1
0
Silver badge

"Would you describe a screen reader as "non-essential stuff"?"

Context is everything.

3
0
Silver badge
Megaphone

"Just about every non-trivial website on the planet loads in resources provided by other companies and organizations"

"Another good demonstration of why ad blockers and script blockers are essential."

And a reason to apply a CLUEBAT to those web site authors that propagate this nonsense, who seem to have NO clue at all...

3
3
Silver badge

"Another good demonstration of why ad blockers and script blockers are essential."

Until someone pops a miner into one of the adblockers - which happened to one of the Chrome youtube adblockers about two weeks ago.

Noscript caught it - but it was difficult to trace as it only attempted activation on some (but not all) non-https sites and never on the popular ones, pointing the finger at those sites instead of the adblocker.

1
0
Silver badge

Re: Cant win...

> Or does that sound too hard?

No, but if that solution didn't bring its own problems, it'd be a common design.

> Use a local proxy to cache all your remotely collected scripts. Have that proxy run a comparison check against the last known good version for all external scripts.

If you locally proxy then you need to deal with all your traffic. Troy Hunt's blog about this story mentions for his site today that this would be 500GB of minified http data just for jQuery if he didn't offload it to cloudflare CDN. That's costly. You can make all sorts of arguments about how people shouldn't use all these libraries, that we should jump right into notepad to develop our static pages, but that kinda misses the whole point.

Another downside of your local cache model is that your performance will be crap if I'm not geographically close to your server. Most of those problems disappear if your are delivering off googakaflare. Even if you put your own site on a CDN, it's still a download overhead to your first time visitors.

You haven't specified how you differentiate a new good version Vs a new not so good version. Your technique tells you when a script changes but so does SNI, and you can do that in 30 seconds.

The problem with local caches or SNI is that they solve this problem, but block solution to the counter problem. Imagine that instead of being pwnd, this library vendor was privately reported an XSS bug that allowed your (and any other site using the same library's customers) private data to be exposed. Without SNI and with everyone referencing a common version, that XSS flaw could be immediately fixed across millions of websites. With SNI and private caches, you can't do patch management in that way. That is the crux of why it is a hard problem.

1
1
Bronze badge
Facepalm

1) If a site I depend on gets hacked, I'm not going to know until they tell me or I read about it here.

2) If a site I depend on gets hacked, I have no tools or ability to fix the problem.

3) If a common site gets hacked, then everyone who depends on it gets hacked, per this example. Since crypto-jacking is about making money, this is a particularly important point.

Of course, 1 can be partially mitigated if I'm running a round-trip test from the outside.

The idea of including off-site scripts that have not been digitally signed (recursively) is nuts. That doesn't fix everything, of course, but it is a start.

1
0

Re: Cant win...

There's no super-perfect solution to this, but a combination of CSP and what you propose (perhaps with a "alert warning" rather than stop script running for the WebAdmin to evaluate, depending on the script?)

1
0

Complexity

To paraphrase something in the current edition of The Econimist, in an article about Elon Musk's SpaceX and Tesla companies):

"There will always be complexity. You can choose to have that complexity outside your business, where you have little or no control over it, or you can bring that complexity in-house, where you can see it, measure it and control it."

2
0

Don't load third-party scripts

Just don't do it. It's not worth it. We're seeing reports now nearly every day that third-party scripts, usually ad platforms, get hijacked and that high-profile websites start dropping malware or running coin miners.

Besides, I question the practice of government websites connecting to third-party domains. If you're running a gov site, security is a top-tier priority. This time we had a script being hijacked for coin miners, but what if it got hijacked by credentials-stealing code? Gov sites deal with highly sensitive information, and as such shouldn't run any code that its maintainers aren't 100% what it does. Concretely, what this means, is that they should host their own instance of the service and serve the scripts from their own domain. That this isn't already established policy amounts to sheer lunacy.

59
2
Silver badge

Re: Don't load third-party scripts

Agree about not loading off-site (3rd party) scripts, or anything else.

The other non-first party resources are frequently trackers which don't belong on a true public/gov't site.

Laziness and perhaps some personal gain are the main reasons that developers link out to 3rd party resources.

When a site relies on these it is also exposing itself to DoS and multiple failures as well as slower loading times.

27
0
Silver badge

Re: Don't load third-party scripts

Banks are equally guilty. They've all opened themselves up to the third party being compromised, their DNS hijacked, or slurping (why the would govt or banks need Google Analytics or fonts loaded from Google or whatever). And then we're all repeatedly taken by surprise when stuff like this happens.

21
0
Anonymous Coward

Re: Don't load third-party scripts

Fine in theory, but in practice this can cause sites not to work correctly.

Solution, keep it in house, audit all third party code and stop being a muppet (yes you so called IT experts out there)

FFS WAKE UP

re

Just about every non-trivial website on the planet loads in resources provided by other companies and organizations – from fonts and menu interfaces to screen readers and translator tools.

17
0
Silver badge

Re: Don't load third-party scripts

Basically agree that govt sites should probably get 3rd-party plugins from a government secure CDN. But even that has problems.

Given the way government IT works, it would take at least nine months, at a cost of £250K to get an updated copy of a JS library installed on the CDN, which is a pain if three new versions have since come out, each of which fixes an important hole.

Be cheaper to write your own library - at least the holes would be your own!

7
0
Silver badge

Re: Don't load third-party scripts

@AC

"Solution, keep it in house, audit all third party code and stop being a muppet (yes you so called IT experts out there)"

I wish I was obscenely rich too!

Although, if I was, would I be wasting my life in IT?)

Code needs to be 'fit for purpose'. If the 'purpose' is e.g. a website to advertise a product that will earn your company £50K p.a. you can't afford code audits of JQuery, Ruby or whatever the current flavour of the month is. If you insist on the audit, we'll either go bust or we'll forget the 3rd party stuff and dig out the old copy of Dreamweaver and hand code it in plain HTML.

Other than taking extreme care of sensitive data, there's no perfect solution.

11
1
Silver badge

Re: Don't load third-party scripts

"And then we're all repeatedly taken by surprise when stuff like this happens."

Who's this "we" you're talking about. A lot of use aren't.

10
1
Silver badge

Re: Don't load third-party scripts

"If the 'purpose' is e.g. a website to advertise a product that will earn your company £50K p.a. you can't afford code audits of JQuery, Ruby or whatever the current flavour of the month is."

If, for want of a proper audit - or reducing the amount of flavour of the month - the consequence is that you end up damaging your would-be customers the loss of reputation, damages and maybe fines is also something you can't afford.

Security may be expensive. Lack of it can cost more.

16
2
Silver badge

Re: Don't load third-party scripts

"A lot of use"

Us, dammit. And dammit to the forum saying it's still in the edit window and then refusing to accept the edit.

8
1
Silver badge

Re: Don't load third-party scripts

So what's the alternative, exactly?

1. Write, maintain and test everything in house. Oh, and remember to document it too, because otherwise you're just storing up trouble for next week. And even then you'll still have dependencies - on browsers, on server platforms and scripting languages - and vulnerabilities will still creep in. I'm not really seeing the business case for that.

2. Make sure every resource is fully audited, and can't be amended without appropriate hoop jumping. This is marginally less work than (1) (and commensurately slightly less secure), but frankly it's still a shedload of effort for very small return.

3. Avoid scripts entirely. Congratulations, now you spend your whole life saying "no" to the marketing department. Good luck keeping your job, even if your company can survive.

Or 4. Accept that the occasional breach is part of your normal operational costs. Just like you expect employees to pinch some of your stationery, you expect customers to duck out on some of their bills, so you also expect hackers to disrupt some of your transactions. Accept it, model it, budget for it. Move on.

7
9
Anonymous Coward

Re: Don't load third-party scripts

@Pen-y-gors

Sure, and when all your customers have gone, or no one wants to use the website/service because its not trusted, your profits will go down and you will go bust.

Don't piss off the customer/client as they provide your wages. ferengi rule of acquisition 4098

9
1
Silver badge

Re: Don't load third-party scripts

@Veti

"Or 4. Accept that the occasional breach is part of your normal operational costs. "

Except that it's your customers who are paying for it. That option bespeaks a really shitty attitude towards the users of your services.

11
1
Silver badge

Re: Don't load third-party scripts

That option bespeaks a really shitty typical industry attitude towards the users of your services.

Our motto is: we don't care, we want the money now, we probably won't be here tomorrow.

10
0

Re: Don't load third-party scripts

'Congratulations, now you spend your whole life saying "no" to the marketing department.'

You say that like it's a bad thing.

11
1
Silver badge

Re: Don't load third-party scripts

"You say that like it's a bad thing."

I've never worked anywhere where the IT department got a veto over anyone, let alone marketing, so saying "no", even to the most wrong-headed idiocy, can cost your job.

I've always suspected that marketing is a lot less effective than the marketeers like to make out, but they are undeniably good at keeping their jobs and budgets.

5
0
Silver badge

Re: Don't load third-party scripts

now you spend your whole life saying "no" to the marketing department.

Definitely a good idea. Every business should appoint someone full time to do just this. Come GDPR time, now only a few months away, they're the ones most likely to bring big fines down on your company.

5
2
Silver badge

Re: Don't load third-party scripts

"but they are undeniably good at keeping their jobs and budgets."

They're going to need their budget if the GDPR fines they bring down are charged to it.

3
2
Silver badge

Re: Don't load third-party scripts

@veti.

5. If it's marketing driving this adopt some combination of 1 and 2 and charge the work to their budget.

3
2

Re: Don't load third-party scripts

> Concretely, what this means, is that they should host their own instance of the service

That messes with Browsealoud's business/licensing model - and it's far from a cheap product. Browsealoud pre-dates the (W3C) Web Speech API which is supported by all current browsers, trivial to implement and should have made the product redundant anyway.

3
0
Anonymous Coward

Re: Don't load third-party scripts

> Other than taking extreme care of sensitive data, there's no perfect solution.

Yup. Reading through these comments you can easily tell the difference between those posters who feel entitled to an opinion on the basis that they read stuff from websites¹, and those who actually write websites.

I found your other comments very sensible too.

¹ Including those of us who code websites as a hobby outside an organisational context.

3
0
Anonymous Coward

Re: Don't load third-party scripts

> Security may be expensive. Lack of it can cost more.

Can you offer any actual examples from your own experience to illustrate the content of your post?

I am curious to know if you really did suggest to your organisation that the entire stack should be audited and analysed before running their latest "buy one get one free" campaign.

Have you ever heard of "ALARP"?

To give you an example from my own field, which is aviation: there are various scenarios during a take-off sequence in which an engine failure may, or inevitably will, depending on the scenario, lead to a crash. As will, rather more obviously, a sufficiently large loss of propulsion at a certain point after the start of the take-off run. We know this. We have actually quantified that risk. What do we do? Do we give up on aeroplanes? Or do we "just" mitigate this to a point where you say "fuck it, beyond this it just wasn't your lucky day".

Happy flying! :-)

1
0
Anonymous Coward

Re: Don't load third-party scripts

> I've always suspected that marketing is a lot less effective than the marketeers like to make out

The larger your marketing department, the less pound for pound effective they are, but make no mistake: good marketing is what makes or breaks your organisation.

A billionaire acquaintance of mine told me very clearly once: "the first employee I hired was a salesman", only the third was a developer (I forget what the second was).

One thing I would recommend to anyone is to take a short marketing course. I found that it really helps to make more sense of our world, and make better buying choices once you can see through the marketer's spells.

1
0

Do we give up on aeroplanes?

"What do we do? Do we give up on aeroplanes? Or do we "just" mitigate this to a point where you say "fuck it, beyond this it just wasn't your lucky day"."

...I'm a scaredicat. Yes, I gave up on planes....and bungi jumping, as well as the 4th rung on ladders :)

...If only I could give up on government websites I'd be a happy boy :)

2
0
Silver badge
Devil

Re: Don't load third-party scripts

"3. Avoid scripts entirely."

you need to sell this better.

1
0
Silver badge
Devil

Re: Don't load third-party scripts

6. Come, look at this marvelous view from the ginormous 3rd floor office window...

2
0

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Forums

Biting the hand that feeds IT © 1998–2018