back to article Despite the headlines, Rudd's online terror takedown tool is only part of the solution

The UK government launched its swish new tool to fight online extremist content to much fanfare this week – leading news bulletins and generating reams of coverage. But it also faced a whole host of criticism and concern. Underneath the claims made by Home Secretary Amber Rudd yesterday, there are many questions yet to be …

  1. Doctor Syntax Silver badge

    "He said that the biz had balanced false positives with performance to tune the algorithm to be able to detect 94 per cent of Daesh propaganda with a 99.995 per cent accuracy."

    Did you ask him what he meant by this? AFAICS if it correctly detects 94% of videos it's 94% accurate. What, then, if anything, is this 99.995% figure? Unless he has a sensible explanation for this then I wouldn't trust any figure he provides. It's simply marketing gobbledgook and deserves to be treated an the same way as any other garbage spewed by marketing mouthpieces.

    1. wolfetone Silver badge

      It detected 99.995% of the 94% material it was given that it should've detected.

      Or, in simpler terms: 60% of the time it works every time.

    2. Anonymous Coward
      Anonymous Coward

      Accuracy = (True Positive + True Negative) / (Positive + Negative)

      1. ibmalone

        ^ that one. Though given the sets outlined in the article I'm not sure they're actually assessing accuracy, it sounds more like the 99.995% figure is specificity. To get accuracy they have to factor in how common positives and negatives are.

        For the original question (and my earlier reply went astray, so this is the even shorter version): there are different figures of interest. Changing one part of your detection may change some more than others, and overall success rate is still dependent on how frequent positives (in this case, terrorist videos) are relative to negatives (cat videos).

        Sensitivity. How frequently the test reports positive given only positives. 0.94something here. You can trivially make it 1.0 by always reporting a positive.

        Specificity. How frequently the test reports negatives given only negatives. 0.9995 here. Trivially 0.0 by always reporting negatives.

        Obviously in most cases you can make a tradeoff of one for the other. Receiver operating characteristic curves are usually used to illustrate that. Flip a coin every time and both your specificity and sensitivity are 0.5 (as is accuracy, which is usually more complex), any random choice that doesn't look at the input sits on a diagonal line on the ROC plot.

        Accuracy is a composite, but a more complex number. As Mycho said, Accuracy = (True Positive + True Negative) / (Positive + Negative). Positive plus negative is just all your samples, but true positive depends on how many positives there are and sensitivity, true negative depends on how many negatives there are and specificity, so your accuracy figure depends on the ratio of real positive and negatives in the input. If the vast majority are negative, then sensitivity has very little effect on accuracy, if most are positive then specificity has very little effect.

        This tends to come up in medical screening, https://en.wikipedia.org/wiki/Sensitivity_and_specificity two numbers not much talked about are positive predictive and negative predictive value. Both of these mean if you select samples (videos, people) from the general population (youtube, a screening population) based on a test, how much does the ratio of positives (terror videos, people with a particular disease) change relative to the population. A rare disease and a test that is not sufficiently specific may still end up selecting a majority of people who do not have that disease, even if sensitivity is incredible.

        1. ibmalone

          Too late to edit:

          Specificity. How frequently the test reports negatives given only negatives. 0.9995 here. Trivially 0.0 by always reporting negatives.

          For specificity should be: "Trivially 1.0 by always reporting negatives". On the flip side, always reporting negative regardless of input gives 0.0 sensitivity, while always reporting positives gives 0.0 specificity.

          1. John Smith 19 Gold badge
            Gimp

            "We need to be very wary of slippery slopes here," said Bernal. "

            Too f**king true, given the Home Office's behavior.

            1. Rusty 1
              Unhappy

              Re: "We need to be very wary of slippery slopes here," said Bernal. "

              Especially when that slope is more accuractely described as an overhang.

  2. Ronome

    You voted for mafia that is what you get...It is just censorship under a different name, and control...

    1. Brewster's Angle Grinder Silver badge
      Facepalm

      There are legitimate concerns. But those are derailed as soon you extremists start making the non-removal of content non-negotiable. Terrorism and hate speech is pissing in the water we all drink; it has no place in society.

    2. }{amis}{
      Unhappy

      I Remember

      I remember that the Blair / Brown government was just as likely to trot out the "But Terrorism" defence every time that introduced yet another piece of intrusive legalisation.

      At this point I am forced to feel that this attitude is endemic in the whole UK government machine and the front bench of the day only puts a face to the mess.

      1. Teiwaz

        Re: I Remember

        At this point I am forced to feel that this attitude is endemic in the whole UK government machine and the front bench of the day only puts a face to the mess.

        I am well passed that point.

        We've not had, nor are likely to, a Liberal party government to test the rule on, and unlikely to be able to test the 'new old labour' current alignment before it gets replaced by the new new labour corporate trojans.

  3. Anonymous Coward
    Anonymous Coward

    The moment they start using this I'm going to get a daesh flag and upload pictures of my cat playing around on it.

    Lets see how accurate it is then.

    Bad stupid form of backdoor censorship that will be abused.

    Edit: On second thoughts I'll just superimpose all politicians speeches onto a daesh flag and report them, eventually this AI will consider all politicians medieval terror bastards and block them from the web.

    1. Anonymous Coward
      Joke

      Re : Lets see how accurate it is then...

      I dunno... Is your cat called meow-qaida ?

    2. Pen-y-gors

      Nasty! I like it - a lot!

    3. Pen-y-gors

      And a sneaky variation - the mediaeval Tory terror-bastards might complain that placing them with a Douche-bag flag implies they support them, so just add 'Down with this sort of thing' as well and they can't really object!

      1. wolfetone Silver badge

        "...so just add 'Down with this sort of thing' as well and they can't really object!"

        Careful now.

  4. Anonymous Coward
    Anonymous Coward

    I assume Google will manually review any flagged videos either for reporting to the relevant authorities or to flag them as not terror related.

    If somebody presses the "nah it's another blood cat wearing a tea-towel" button, surely that video will go in to training data further improving the algorithm.

    This is only a problem if Google don't manually review or if further training doesn't improve performance.

    1. Yet Another Anonymous coward Silver badge

      This is only a problem if Google don't manually review

      And the people doing the manual review read Arabic and are experts on middle eastern politics?

      Or is "review" just flag anything with curly writing in it?

      Suppose you were a reviewer for Egyptian Google, could you tell the difference between a Britain First rally and a Remberance Day parade or a Scout troop? Could our algorithm?

      1. Wensleydale Cheese

        "Suppose you were a reviewer for Egyptian Google, could you tell the difference between a Britain First rally and a Remberance Day parade or a Scout troop? Could our algorithm?"

        And what chance does any algorithm stand of coping with satire?

        1. Yet Another Anonymous coward Silver badge

          Facebook's child porn algorithm blocked the picture of the Vietnamese girl whose clothes had been burned off by Napalm and their manual review confirmed the decision.

          Or perhaps it wasn't their child porn algorithm, perhaps it was their corporate media profile algorithm responding to their SLA with Dow Chemical ?

      2. SolidSquid

        This is my thinking. Daesh doesn't need to bypass it so much as make it unusable by getting so much material flagged as extremist it becomes useless. And that's not factoring in trolls who would do that just for the hell of it

  5. Anonymous Coward
    Anonymous Coward

    Solution to what?

    Long term content censoring of things the gov't deem "Un-British." (Like porn?)

    1. H in The Hague

      Re: Solution to what?

      "Long term content censoring of things the gov't deem "Un-British."

      Errm, I thought censoring was considered unBritish - or did I (or Amber Rudd) get things in a muddle?

      Confused now.

      1. Anonymous Coward
        Anonymous Coward

        Re: Solution to what?

        Wait til they're deporting us all to Ireland for watching EWTN.

  6. Zippy's Sausage Factory
    Black Helicopters

    Appeals process?

    No doubt, there won't be one. At least, not one that matters. My guess is it might look a bit like this:

    "To appeal, send a letter to this obscure government department (email not accepted). Within 30 days we will reply whether to allow the appeal. You then have six weeks to submit any further information for the appeal, and we then will have a target to decide 25% of all appeals within six months of the original takedown date (up from 18% last year). Please note all appeals are under an NDA and it's a criminal offence to reveal whether you have appealed or what stage the appeal is currently at, even when we've decided in your favour (which, let's face it, we probably won't)."

    1. Yet Another Anonymous coward Silver badge

      Re: Appeals process?

      even when we've decided in your favour (which, let's face it, we probably won't

      Please also note that if we refuse your objection you will be charged with supporting terrorism.

      Or just quietly go on a secret list of terrorist sympathizers.

  7. Anonymous Coward
    Anonymous Coward

    OK, who's for getting a copy, training it with party political broadcasts and requiring broadcasters use it?

    Not easy I grant. Propaganda is propaganda.

    Then if party political broadcasts can be filtered maybe move on to adverts.

  8. dbgi

    And so the "terrorists" come up with ways to get round the filtering

    Like all government policy, ease it in slowly with moral reasons and people won't notice the subtle changes or care. Then before you know it, it's too late to object.

  9. Anonymous Coward
    Anonymous Coward

    I suppose on the plus side they'll be less hipster videos.

  10. macjules
    Black Helicopters

    Uploading Policies?

    Dare I presume that uploaded videos must not include acts of beheading, pleas to 'come and work for ISIL, the friendly international terror organisation' or other Daesh propaganda .. and doesn't that rule out most Daily Mail uploads as well then?

    On the presumption that most video content from Daesh is going to be rich in violence, rhetoric and the promise of paradise surely the ASI logarithm has a pretty clear criteria to start with, and therefore the 99.99etc% reliability claims are indeed sound?

    1. Anonymous Coward
      Anonymous Coward

      Re: Uploading Policies?

      'On the presumption that most video content from Daesh is going to be rich in violence, rhetoric and the promise of paradise...'

      Also be a bit of a bugger for anyone quoting from the Old Testament.

  11. Mystereed

    Whack-a-mole?

    1. The machine learning algorithm gets trained using an existing set of known styles of video.

    2. Deploy to live and start rejecting new videos which match criteria (I assume maybe the accounts also get flagged as potential producers of dodgy material and get suspended, rather than having special forces sent to where the user is logged on?).

    3. Dodgy uploader uploads their latest video using fresh account.

    4. Dodgy uploader gets told their video has been rejected.

    5. Dodgy uploader contacts their film director/editor to tell them their latest style has been compromised.

    6. Dodgy director/editor moves on to a new style, and gives video to uploader. Go to 3

    7. When a human sees an example of new style, reports it. Goto 1

    8. So eventually, based on the above rules, we will have every publishing/film style ever used by these people? Black & White silent movies, French New Wave, Manga, etc. etc. and then they will have to give up? Is that the theory? Looking forward to their version of Carry on up the Khyber btw.

  12. Anonymous Coward
    Anonymous Coward

    If this tool is so great why don't they use it to find all the videos already posted online?

    Answer: because it's not.

    1. David Roberts
      Holmes

      Already posted?

      That will be the initial training and trial. Working on historical data. Once a few passes with refinements added manages a very low hit rate from undetected videos then the fun starts,

      Full scale whack-a-mole.

      This just raises the bar so that a 2 minute amateur video from a cellphone now gets screened out and more effort is needed to publish.

  13. Anonymous Coward
    Anonymous Coward

    Frankly

    I'm disappointed.

    Watching one group of religious idiots killing off ANOTHER bunch of religious idiots is a highlight of my weekend.

    Still, I'm sure TOR will still allow me to view my videos of choice at a time and place I choose to do so.

    Sorry, not sorry.

    1. Alistair
      Windows

      Re: Frankly

      @cornz:

      Yes, but the NFL is broadcast live on TV, you don't need the internet or tor for that.

  14. Chronos
    Flame

    Aren't we forgetting something?

    It was this rabble's predecessors and their mates abroad who started the whole thing in the first place. Listen to We Didn't Start The Fire by Billy Joel for some examples of what a foreign policy which has a Geordie Saturday Night attitude to someone spilling their allies' pint does to the world.

    Yet, somehow, it's the ordinary people who must sacrifice any illusion of privacy and free speech while having their genitals inspected at airports by an ape with a three digit salary and an IQ that begins with a decimal point to clean up the mess they made and are probably going to continue making with impunity. Bastards.

    The "how they're going to tackle this" isn't quite as important as the "why they now have to" which should, all things being equal, add a little perspective to the debate. Of course, it won't.

  15. mark l 2 Silver badge

    I am guessing the creators of this AI have not checked out some of the more nefarious black hat forums on the internet where there are entire threads on how to bypass the copyright filters used by Youtube and the like to upload copyrighted content without it getting flagged.

    All that will happen now is that ISIS will start to use those black hat techniques to fool the AI and by pass the checks, resulting in ISIS videos making in through while lots of false positives will be held for moderation.

  16. Anonymous Coward
    Anonymous Coward

    We wouldn't have ISIS if it wasn't for Blair and Bush invading the wrong country in the first place....

    1. Yet Another Anonymous coward Silver badge

      We wouldn't have ISIS if Saudi Arabia hadn't lost control of Al Queda over a disagreement over why the targets should be chosen by America, rather than be America.

      And we wouldn't have that if it wasn't for Iran having a leadership Saudi Arabia don't like.

      And we wouldn't allow that unless they also had a leadership we didn't like.

      We they wouldn't have if we hadn't deposed the leader before last and imposed our own mad dictator

      And we wouldn't have that popular leader if we hadn't run the country as our personal oil field

      .....

      And we wouldn't have that if Ug the caveman hadn't worshipped the moon in the wrong way according to Blurg the caveman.......

      1. Chronos

        Ah yes, the age-old "my god has a bigger dick than your god" issue. For all the rhetoric about separation of church and state, the disorganised religion mob do still tend to sway the decision-making process quite a bit while, it must be pointed out, paying none of the tax to actually implement those decisions.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like