back to article Boffins build AI that can detect cyber-abuse – and if you don't believe us, YOU CAN *%**#* *&**%* #** OFF

Trolls, morons, and bots plaster toxic crap all over Twitter and other antisocial networks. Can machine learning help clean it up? A team of computer scientists spanning the globe think so. They've built a neural network that can seemingly classify tweets into four different categories: normal, aggressor, spam, and bully – …

  1. Anonymous Coward
    Anonymous Coward

    Fast forward to a treehouse of horrors dystopian future where the goody two shoes have take over the asylum, if you're caught using naughty words it's off to re-neducation for you.

    I didn't hear a diddly from you.

    1. AMBxx Silver badge

      We're there already - try having a non-liberal viewpoint on pretty much anything.

      1. Anonymous Coward
        Anonymous Coward

        We're there already - try having a non-liberal viewpoint on pretty much anything.

        If I disagree with you, will I get downvoted? ;-)

      2. Ben Tasker

        I disagree, I hold lots of view points that are "non-liberal", and generally don't get much criticism for them - the odd bit of robust debate, but not criticism, and certainly not personal criticism.

        There are certain... ahem... "non-liberal" viewpoints I neither hold, nor express. Those are the ones I tend to see people being criticised for, usually because they rely on taking away someone else's rights, or agency over their own bodies/lives.

        What I'm trying to get at here, is if you're personally being criticised, it's probably not because what you're saying/doing is "non-liberal" but because it's actually objectionable in some way. Obviously there are exceptions to every rule.

        But it's far easier to blame others because "liberals" rather than to look at your own stance/position (and to some extent, those you're standing alongside) and understand why people might find them objectionable.

        1. Clunking Fist

          "usually because they rely on taking away someone else's rights, or agency over their own bodies/lives"

          (I'm kind leaning somewhat pro-choice myself, but:) Yes, now dare you apply a mild smack to your child for the purpose of behaviour correction, or gender them at birth. But feel free to kill them before birth if their existence will be a bother. Etc.

          1. Ben Tasker

            > now dare you apply a mild smack to your child for the purpose of behaviour correction

            That's about the child's rights, not yours - so it's not taking away your rights, because you don't have the right to strike anyone else. Much like the upset at not being able to ban abortion, it's upset at being told you can't infringe someone else's rights.

            I'm trying, really hard, here not to offer an opinion either way on smacking because it's the underlying principle that matters.

            I would say though, that the majority of objection I've seen (and certainly objection by law) is not about "mild" smacks, but about being excessively rough. Not to say there aren't those that oppose *any* form of smacking.

            As for objections to gendering at birth, that is pushing it too far IMO, but it's also far from the mainstream position - even on the left. When the child is older, then it's their choice - though I can understand a parent struggling with this, even out of habit - but I suspect it's potentially just as harmful to deny gender at a young age.

            In much the same way, I don't agree with giving children drugs to block puberty - they're too young to understand the ramifications of that.

            There was a video I saw recently (IIRC it was a US reality show at that) where a teen M -> F was talking to a doctor about the forthcoming gender-change op. She asked about whether she'd have much "depth" after the change, and was told that because of the puberty blocking meds she'd been taking, her penis was under-developed and therefore there'd be maybe a couple of inches depth at most - not nearly enough for comfortable penetrative sex.

            So, despite having the very best of intentions, the group that gave her those puberty blocking pills have created a new issue - and one that will be of increasing importance as she grows older, sex becomes quite a big part of adult life (at least for a while).

            In no way is this to say that she shouldn't be allowed the op, or to live her life as she sees fit, but it was entirely irresponsible of those in a position to do so to have given her those pills, especially given it seems she wasn't told about an entirely forseeable drawback of doing so.

            Now there are undoubtedly people who will disagree with my assessment of this situation, or even just the position I've drawn as the result of a pretty small sample. But the number of people who will complain that you've announced you've got a baby girl? Pretty damn small (well, unless the baby is in fact a biological boy).

      3. Anonymous Coward
        Anonymous Coward

        20 years ago:

        spAmm0r_xxx: "Women are stupid!"

        femaLe_32: "look at that idiot.."

        us0r_321: "what a clown"

        un1corn: "lets ignore that troll"

        Today:

        spAmm0r_xxx: "Women are stupid!"

        (This post was tagged, you are getting hellbanned and sentenced to 3 years in Reddit)

    2. Chronos

      Oh, for goodness' sake, use your intelligence! A well placed insult is undetectable in most cases.

      "Congratulations on the new job. They'll be very lucky to get you to work for them."

      1. stiine Silver badge
        Thumb Up

        Get that often do you?

        1. Chronos

          See how easy that is? Well done.

          1. 's water music

            you must come round for dinner some time

            1. Chronos

              Sorry, I'm a Big Yin fan. I already know that this translates to "you're a boring bastard, I'm off."

      2. Stoneshop
        Devil

        And then there's diplomacy

        Telling someone to go to hell[0] in such a way that they'll be looking forward to the trip.

        [0] 63.4444°N 10.9227°E. Trøndelag, Norway. Half an hour from Trondheim by train.

  2. Anonymous Coward
    Anonymous Coward

    Understanding sarcasm is difficult,

    Ah, so you're an American then...

  3. SonofRojBlake

    "Understanding sarcasm is difficult"

    Yes, of course it is.

  4. sum_of_squares
    IT Angle

    I don't buy it. There are so many different ways to be rude, no way any AI could possibly detect all of them.

    I mean how do you train such a thing in the first place? All those nuances of human language allow for very sublte insults (ie. "get that [censored] out of your [censored]!" or "Wow, your mom is such a nice [censored], I'd love to [censored] her [censored] one day!"). I doubt a [censored] AI will ever be able to detect that.

    Also all those oh-so-smart [censored] who think they could make the internet a nice and safe place without sacrificing freedom of speech are just [censored] to me. Seriously, just [censored] yourself and all of your [censored] [censored], you [censored]!!

    EDIT:

    Wow, what was that?

    1. Just Enough

      If you can understand the different ways to be rude, then there is no fundamental reason why AI can't. It certainly isn't easy for AI, and may not be something it's capable of at the moment. But suggesting it'll be forever beyond its capabilities is nonsense.

      1. Anonymous Coward
        Anonymous Coward

        "If you can understand the different ways to be rude, then there is no fundamental reason why AI can't".

        I wish I could understand all the different ways to be rude.... I just know that one day I'm going to say something and not realise I've insulted someone.

        EDIT:- Perhaps I've done it already?

    2. SW10

      There is a serious point here about how the neural network is trained.

      If it’s a group of (say) white blokes tagging what is ‘normal’, ‘bullying’, ‘aggressive’ etc., they could well overlook or miscategorise things that are perceived as abusive (or banter) by others.

      1. Clunking Fist

        "(or banter)"

        Indeed, for a period, a bunch of us referred to each other as fucktards, with numbers. I was fucktard 3 from memory. Other folk found it alarming when they first stumbled across this, but got used to it fairly quickly: I think it was clear that we were just a bunch of silly buggers.

  5. Anonymous Coward
    Anonymous Coward

    a system that can automatically and fairly

    ah, that would be at least as good as the youtube censorship system, eh?

    1. Anonymous Coward
      Anonymous Coward

      Re: a system that can automatically and fairly

      And before that, the AOL/CompuServe forum censorship system that banned cancer survivors from mentioning certain body parts

      (I had a friend, a breast cancer survivor, affected by this... no fun trying to find an acceptable substitute)

      1. Stoneshop

        Re: a system that can automatically and fairly

        A tumor of the female endocrine glands, or, per Frank Zappa, her mammalian protuberances?

      2. J. Cook Silver badge

        Re: a system that can automatically and fairly

        Yep. that is just clbuttic.

      3. jake Silver badge

        Re: a system that can automatically and fairly

        AOL banned entire cities. Scunthorp and Penistone are two that come immediately to mind. There were others.

  6. Anonymous Coward
    Anonymous Coward

    The aim?

    "The aim is to create a system that can filter out aggressive and bullying tweets, delete spam, and allow normal tweets through"

    It'll have to be a damned good system if it was to be relied on to censor people's feeds (whether they choose it or not).

    I'd prefer to see it used to tag posts - sort of a 'public shaming'. If nothing else, they really need to put it out there in that form in order to get feedback on how well it is doing.

    Perhaps El Reg could offer to be a guinea pig (when it moves beyond twitter) - offering up the comments sections on a few suitably contentious subjects.

  7. Arthur the cat Silver badge

    Obligatory xkcd

    Here.

  8. druck Silver badge
    Coat

    AI...

    ...making the world safe for snowflakes everywhere.

    1. Chris G

      Re: AI...

      Snowflakes aren't the problem, the problem is the people who are trying to protect us from ourselves and suffocating freedom of speech in the process.

      Who gets to decide the limits of what can be said, to whom and what context, particularly on the occasion s when something may be unpleasant and/or aggressive and may even be in a bullying tone but is a necessary truth.

      Nobody has to like what I say but I do have the right to say it.

      Disclaimer:

      If anyone is offended, outraged or feeling oppressed by this comment, someone made me say it.

      1. stiine Silver badge

        Re: AI...

        Who gets to decide the limits of what can be said, to whom and what context,

        That would be the speaker.

      2. Clunking Fist

        Re: AI...

        "If anyone is offended, outraged or feeling oppressed by this comment, someone made me say it."

        Surely you mean "I misspoke"?

  9. Michael H.F. Wilkinson Silver badge

    I am sorry Dave, I cannot post that for you

    And my name isn't even "Dave"

  10. Pete 2 Silver badge

    What did you call me?

    > classify tweets into four different categories: normal, aggressor, spam, and bully

    Given that half of americans can't tell when a Brit is calling them an idiot, I don't hold out much hope for this.

    1. Anonymous Coward
      Anonymous Coward

      Re: What did you call me?

      Yes but they're not normal. Bully

      You're not normal. Aggressor

      If you want to be normal click this link. Spam

      I bet AI could work out the point of this comment. Sarcasm -> Crash, which is normal I guess.

    2. ZenCoder

      Re: What did you call me?

      Very interesting commentary, I'll bare in in mind.

    3. jake Silver badge

      Re: What did you call me?

      The other way of looking at that is that the Brits can't communicate effectively outside their own little insular community.

  11. Anonymous Coward
    Anonymous Coward

    influencers

    "Spammers, on the other hand, are less likely to use abusive language and tend to sell things like smut pics and videos"

    Or 'influencers' as they now call them

  12. sabasadegh1

    I suggest to use https://lookup.tools to get whois for all GTLD and CCTLD and also access to free Phone, NS, IP, Email address and owner reverse whois.

    1. Clunking Fist

      So dox someone when you disagree with them?

      1. David 132 Silver badge

        Sssh. Don’t dignify the spammer-bot with a response.

    2. Clunking Fist

      Criticism of Diane's Abbott's ideas is categorised as "attacking a wonderful woman of colour", whereas criticizing Priti Patel's ideas is bashing a nazi, etc.

  13. liac

    Why not....?

    ...use AI to help people who are 'sounding' fragile, hurt or making self-harm comments.?

    trolls etc.. will always resurface...not so much the hurting ones..

  14. Anonymous Coward
    Anonymous Coward

    Hmm, some interesting questions floating around

    Let's start with the basic one: we know that certain tribes are not terribly good with the whole sarcasm/irony thing, so how can we expect an AI to do better? Next, are we aiming at 100%, or do we deem it acceptable that 20% will be misclassified? And who will do the tuning of the misses, or is that where the humans come in again? (in that case, forget it, the volume merchants are exactly in this game to exorcise the humans from the chain because they get too much in the way of profit).

    I'm not terribly confident this will deliver, but it's interesting to see them try. I still think that AI is no match for HB (Human Boneheadedness).

  15. spold Silver badge

    Important to note...

    The job of these technologies is primarily to identify potential policy transgressions that have not already been flagged by reports of abuse...

    The job of a content moderator is to ascertain to whether flagged content (in either context) does or does not meet POLICY.

    So the technologies cannot be left to enforce arbitrary evaluation... and certainly not application (unless you are into automatic mass censorship).

    The self-protecting restriction of a content moderator is not to apply their own values to moderation decisions, but to apply policy rigorously to the best of their ability, policy is decided by the content owner/facilitator.

    AI will be important in potential identification (possible unreported policy violation and sliding down a slippery slope into application based on huge volumes).

    Yours, Dick Babcock. (Buy my book)

  16. Wellyboot Silver badge
    Flame

    Will this work with Euphemisms?

    If not they might as well place the AI where the sun don't shine .

    Icon for curry lovers.

  17. Tromos

    Obfuscation could be the key

    Throw in a few positive terms like 'grand' and 'master' and I bet the AI let's you get away with calling someone a big w*****.

    "A master baiter of grand proportion."

  18. sbt
    Trollface

    Whoa, hang on there! Do not mess with the tabulator.

    Call me a nazi-paedo-scientologist if you like, but not a spacer.

    1. jake Silver badge

      Re: Whoa, hang on there! Do not mess with the tabulator.

      Feelthy TABs are the devil's work, unless you are using them on your Smith Corona.

  19. jake Silver badge

    One wonders what would happen if you invited the bot herders to ...

    ... re-tool their entire classification system.

    English is wonderfully flexible when it comes to insults :-)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like