back to article UK Home Sec Amber Rudd unveils extremism blocking tool

UK Home Secretary Amber Rudd has announced a tool that purports to detect and block jihadist content online, and tech companies may end up being legally required to use it. London-based firm ASI Data Science was handed £600,000 by government to develop the unnamed algorithm, which uses machine learning to analyse Daesh …

Silver badge

Well this is fantastic. I guess we've entered the age where the Government will be able to censor material from the public without the public ever really knowing.

All in the name of terrorism and our safety of course.

148
3
Silver badge

Or rather what we have actually done is fund another private firm whose owners/founders *know* people in the civil service (and in the space of a couple of weeks this will come out ), along with independent tests showing just how crap this 'amazing' tech actually is. Rudd has put her name to it so we should be pretty sure of the outcome.

Wonder how much from Al Jazeera will get incorrectly flagged?

87
2
Anonymous Coward

Incorrectly flagged?

Better. I'm waiting for everyone holding up random coloured and shaped bits of paper in the videos to break the algo and slip through.

(see the reg articles on breaking ai or image recognition with a single pixel)

33
1
Silver badge

The department claimed the algorithm has an "extremely high degree of accuracy", with only 50 out of a million randomly selected videos requiring additional human review.

1) 'Randomly Selected' - Permanent Undersecretary of State's Pornhub bookmarked favourites?

2) '50 out of a million requiring additional human review' - there were some Pornhub videos he hadn't watched yet?

3) 'extremely high degree of accuracy' - CivilServantSpeak for 'Capita says they have fixed the bugs'.

53
0
Anonymous Coward

"Wonder how much from Al Jazeera will get incorrectly flagged?"

I'm pretty sure at some point that to will become a "terrorist" news outlet.

Can't have the other persons point of view after all.

49
1
Anonymous Coward

False Positive rate ?

Probably includes anything that involves criticism of the current lot or videos of former Home Secretaries in French Maids' outfits.

21
0
Silver badge
Headmaster

I guess we've entered the age where ...

Where have you been this past decade?

Of course I agree with the point you were making. It's your wording I take issue with.

10
0
Silver badge

Quis custodiet ipsos custodes?

27
0

This post has been deleted by its author

Anonymous Coward

Come on. It's progress. They used to burn books.

7
0
Silver badge
Coat

"They used to burn books.

IRTA "They used to bum books."

5
0
Silver badge

Actually it is not 'used' by the Government at all. the purpose of the software is to intercept the upload process and prevent the video actually making it online to Vimeo, Youtube etc., whom I presume would be requested to integrate the ASI software into their sites.

More information here

4
6
Silver badge

Doubtless this has been independently verified and its training set as well as the algorithm are open for inspection.

17
0
Anonymous Coward

If you can do it for that dataset what's to stop it being used for other datasets...a slippery slope...TMay blocking anti May sites...just imagine no May day...no distress calls.

18
0
Anonymous Coward

Oops... sorry we forgot to mention the trojan that was included for free

We would like to use this blocking tool and it does nothing else honest :)

9
0
Silver badge

@wolfetone

Well this is fantastic. I guess we've entered the age where the Government will be able to censor material from the public without the public ever really knowing.

Yes, the slippery sloop has been trod upon and there's no telling how steep it is or how slippery. What's next... political parties added to the mix? Anything on the whim of government wanting to ban? I'm surprised a certain leader hasn't jumped in on this about "fake news"....

8
0
Anonymous Coward

Re: Oops... sorry we forgot to mention the trojan that was included for free

> We would like to use this blocking tool and it does nothing else honest :)

It's more appropriate with the missing comma after "else" !

5
1
Bronze badge

> Wonder how much from Al Jazeera will get incorrectly flagged?

The ASI blurb video on YouTube appears to be using an Al Jazeera news item about Daesh to show the negative response to the scoring algorithm versus a positive response to a propaganda video.

0
0
Anonymous Coward

> "used to bum"

How did you train your brain in such a way? I shudder to think

1
0
Silver badge

@macjules - What really impressed me about that presentation is that they already have a coffee mug branded with their platform name. This must be mature, reliable technology, right?

4
0

Bloom filters at Stainsbury's are $600k and 0.00 01 BTC, that's the way it is.

I wonder how much from Albertson's will get incorrectly flagged.

This will probably put the lid on the Theresa May bits where she wonders about shorting those people engaged in Labor; just when the exact opposite of what it supposedly discusses (plus honorifics and NHS LSD microdosing ad libum for the people who voted stay, and not waiting to TKO the rest of the wait or EU exit or probably poorly coined political terms in general) is earnestly needed.

1
0
Anonymous Coward

Re: False Positive rate ?

>former Home Secretaries in French Maids' outfits.

As an equal-opportunities person, I'm happy to say that that prospect appals me, regardless of the gender of the former HS.

3
0
JLV
Silver badge

>Quis custodiet ipsos custodes?

colere patronos

Their campaign contributors, who else?

1
0
TRT
Silver badge

"unnamed algorithm"

I don't believe it.

£600,000 = £500,000 for the fact-finding market analysis and spin team, £90,000 for cost over-run protection and £10,000 for actual work. The very least they are going to get out of it is a decent acronym to name their tool with.

45
1
Bronze badge

Re: "unnamed algorithm"

I can suggest names for free. "Big Brother" or "Maybot".

38
1

Re: "unnamed algorithm"

I was going to go with "Rudd's Crud"

32
0
Silver badge

Re: "unnamed algorithm"

How about ESPY? Expert System Probes You.

8
0

Re: "unnamed algorithm"

Ruddy hell!

16
0
Silver badge

Re: "unnamed algorithm"

CensorBot.

Soon to be forked by every other Government Department, Quango and busybody who thinks they are entitled to block random content regardless of the actual legislation. (CEOP, FACT, Movie Pigopolists, City of London Plod - I'm looking at you)

The Capita cubicle slaves in some dodgy offshore tech hub are going to be busy operating the master blacklist that it really runs off.The "AI" is just a randomiser on the top.

27
0
Silver badge

Re: "unnamed algorithm"

I can suggest names for free. "Big Brother" or "Maybot".

The name Maybot is already in use.

Sadly, I found Maybot to be rather unconvincing. It made far too much sense and showed too much human feeling to pass for the real thing. But with further development it can only improve.

10
0
Anonymous Coward

Re: "unnamed algorithm"

They could name a boat after it, the censorship or they could call it ruddy mc c*ntface

15
1
Silver badge

Re: "unnamed algorithm"

"I was going to go with "Rudd's Crud""

Comprehensive Restriction of Upload and Download by DAESH

or CRUDD

14
0
TRT
Silver badge

Re:[ "unnamed algorithm"].... I was thinking more

Automated

Detection of

Extremist

Propaganda

Transmission

There you go. Snappy, took 2 minutes to come up with, apt... belies the reality. That'll be £6 million please. More likely:

British

Internet

Governance

Jihadi

Information

Zerg

Upload

Monitoring

7
0

Re: "unnamed algorithm"

The name Maybot is already in use.

John Crace got there long before

6
0
Silver badge

Re: "unnamed algorithm"

As for the project logo, may I suggest a picture of smugly grinning Stalin?

http://www.onlygossip.net/public/upload/images/2/1/b/54540a22ea45a_.jpg

6
0
Silver badge

Re: "unnamed algorithm"

...by DAESH

That's ok, I like my extremism in english, so I go looking for ISIS.

0
0
Silver badge
Joke

detects 94 per cent of Daesh propaganda with 99.995 per cent accuracy

60% of the time, it works every time.

63
1
Silver badge

Re: detects 94 per cent of Daesh propaganda with 99.995 per cent accuracy

What does that actually mean? Either it detects something as what it's looking for or it doesn't. If it detects 94% than that's a meaningful figure. But what does "with 99.99f% accuracy" mean? Unless it's a means of saying it has 0.005% false positives - which they could say more explicitly - I can't see that it has any meaning at all. I would instinctively distrust anyone who produces a statement like that. OTOH I suppose there might have been something meaningful that went into the Rudd "I don't really understand it but it went something like this" regurgitation mill.

I wish she and Davis would swap jobs. He seems to have his head screwed on right about the Home Office and its doings while she seems sound on Brexit.

17
1
Silver badge

Re: detects 94 per cent of Daesh propaganda with 99.995 per cent accuracy

I read that as "it has a 6% false negative rate, and a 0.005% false positive rate". It's expressed clumsily, but it's weighted towards false negatives, which is a good thing.

3
0

99.995% is impossible

Even humans aren’t that accurate.

If you’re positively identifying terror videos at that rate, you must have a huge false-positive rate. So a movie like Mad Max Fury Road, featuring fighty young people with big guns on dusty desert roads, would probably be flagged as jihadi propaganda.

72
3
TRT
Silver badge

Re: 99.995% is impossible

Peppa Pig dubbed in Arabic

44
0

Re: 99.995% is impossible

You laugh but an episode of Peppa Pig is banned in Australia...

(for saying spiders aren't dangerous, basically, which is a little bit misleading in Oz...)

53
0
Silver badge

Re: 99.995% is impossible

"You laugh but an episode of Peppa Pig is banned in Australia...

(for saying spiders aren't dangerous, basically, which is a little bit misleading in Oz...)"

More fool them for living in a country that's trying to kill them every day.

60
0
Silver badge

Re: 99.995% is impossible

> 99.995% is impossible

Not at all impossible for the test's they'll have run.

If you've _very_ carefully curated your test content, with an eye to claiming a high headline effectiveness rate you could quite easily score damn near 100% (though you don't want 100, because people would question that). Of course, the Government would *never* massage figures, so it couldn't possibly be they're using the best result from a test designed to prove it works (rather than look for failure scenarios)

In real world conditions, definitely not going to happen.

40
0
Anonymous Coward

Re: Peppa Pig dubbed in Arabic

The sad part is that they never get to taste it.

32
2
Silver badge

Re: 99.995% is impossible

"Even humans aren’t that accurate."

That's not entirely fair. Given the, presumably deliberate, lack of details given, it's not unreasonable to assume that the test set is actually the same as the training set. Under those circumstances, it's very easy to get close to 100% accuracy - "Here's a video we've already told you is a terrorist video. Is it a terrorist video?". That's about what I'd expect if you pay just £600k to a largely unheard of small company (they don't even get a mention on Wiki, and their own website mainly talks about empowering your productionising (yes, really) by teaching you data science in 8 easy step). You get what you pay for, and Rudd paid for people who can spout buzzwords she likes the sound of.

44
0
Bronze badge

Re: 99.995% is impossible

If they're using the term "accuracy" correctly (proportion of true negative and true positive out of all all outcomes, taking into account relative frequencies of positives and negatives), then it's pretty impressive, though still means about 5 in 100,000 other videos will be incorrectly flagged. All it needed was someone to spend £600k? Bit sceptical, unless it's checking whether the publisher is listed as "Islamic State".

14
0

Re: 99.995% is impossible

You don't have to be that careful. Just use the learning dataset as a test dataset.

22
0
Silver badge

Re: 99.995% is impossible

Here's a video we've already told you is a terrorist video. Is it a terrorist video?

TBH this isn't how that works. It's here's a curated set of ISIS/not videos. Here's another *different* set of ISIS videos mixed with other, which are/aren't, what's the accuracy rate? If they did it the way you say their reputation would be in tatters by the end of the day, I doubt they're that stupid.

That being said with real world data the reality is it's going to be massively inaccurate. It's entirely possible for a tool like this to exist and kinda work but I'd suggest only using it for flagging videos for a real person to look at - although as others have pointed out meat-based AI fails at this sort of thing too.

2
3
Silver badge

Re: 99.995% is impossible

"If they did it the way you say their reputation would be in tatters by the end of the day, I doubt they're that stupid."

Weellll....it is only lunchtime, we've still got some time to play with until the end of the day.

25
0

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Forums

Biting the hand that feeds IT © 1998–2018