back to article Quit worrying about killer robots, they are coming whether you like it or not – and they absolutely will not stop

The use of fully automated AI systems in military battles is inevitable unless there are strict regulations in place from international treaties, eggheads have opined. Their paper, which popped up on arXiv [PDF] last week, discusses the grim outlook of developing killing machines for armed forces. The idea of keeping humans in …

Page:

  1. Zog_but_not_the_first Silver badge
    Unhappy

    You've nicked...

    All my supposedly witty comments in your headline and sub-headline.

    I suppose the only thing left is to take off and nuke it from orbit.

    1. Adrian 4 Silver badge
      Devil

      Re: You've nicked...

      @Zog

      You forgot 'It looks like you're starting a war. Would you like me to help you with that ?'

    2. Fungus Bob Silver badge
      Terminator

      Re: You've nicked...

      "All my supposedly witty comments in your headline and sub-headline."

      El Reg is out there! It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop... ever, until your will to live is dead!

  2. Blockchain commentard Silver badge

    Since when did international treaties get observed by hostile nations? Or by nations losing a conventional war?

    1. Reader2435

      When the other side has the same type of weapons ready to go...

      1. I ain't Spartacus Gold badge

        Not always even then. The tech will get used where it's felt that the advantage of using it outweighs the advantage of banning it. And even then, breaking the ban is always an option. Tech doesn't get uninvented, just because it's banned.

        And even when banned, there are always worse options. For example, in 1990 France announced during the war to kick Iraq out of Kuwait that they considered chemical weapons a weapon of mass destruction - and so as they didn't have any of their own to retaliate with they would be force to fall back on their nuclear umbrella.

        Also it's much harder to ban what you can't verify. Strategic nuclear weapons are quite big and easy to spot. And if you can hide the specific weapon, you can't hide the infrastructure and manufacturing.

        AI is just another kind of software. Is that server rack in your missile system running normal control software or some sort of AI? Now prove it. Erm?

        Not that I believe any kind of useable AI is even remotely plausible in the next few years, so it's a moot point. And certainly nothing you'd trust to do more than offer assistance.

  3. Sgt_Oddball Silver badge
    Terminator

    Has anyone...

    Spoken to the Norks about these treaties? Iran too (to a lesser extent)?

    Both the American and Russians have developed chemical weapons under the guise of using them on civilians (all perfectly legal and above board) so there's always ways around these things.

    Maybe we should also be looking to give these things emotions to aid in trying to avoid being overly hostile or is that just human hubris wanting something that understands consequences?

    1. Dave 126 Silver badge

      Re: Has anyone...

      Companies developed tear gas etc al for warfare *before* chemical weapons were outlawed in warfare by international treaties. It was then that they lobbied their respective governments to use them as civilian 'crowd control' measures.

    2. Bogle

      Re: Has anyone...

      > Spoken to the Norks about these treaties? Iran too (to a lesser extent)?

      Given their haxor teams' experience I think they may just try turning these AIs on their makers.

    3. Kernel Silver badge

      Re: Has anyone...

      "Spoken to the Norks about these treaties? Iran too (to a lesser extent)?"

      Never mind them - there's another country that fancies itself as defender of the world which has a history of not signing up to treaties intended to make war more 'humane', along with a reluctance to hold its military to account for the occasional events that most other countries consider to be war crimes.

      1. Sgt_Oddball Silver badge

        Re: Has anyone...

        Another country? I can think of at least 5 that could be applied to. You're going to have to be more specific...

  4. Persona

    Nearly 50 years too late.

    Target acquisitions systems from back in the 70's were already shooting down unidentified aircraft and on the ground attacking mortar's were being targeted by counter batteries.

    1. BebopWeBop Silver badge

      Re: Nearly 50 years too late.

      And the RAF lost a Tornado in the gulf due to what appeared to be an automated system failing on its IFF

  5. BebopWeBop Silver badge

    “I think that if it can be shown that implementing AI in weapons systems, even in a comparatively simple 'human in the loop' case, creates inevitable pressures to full LAWS systems, that nations may be interested in avoiding an expensive arms race that would produce questionable value.

    They only need to be cheaper/more effective for them to be adopted without thinking through the consequences

    1. thosrtanner

      Don't even need to be more effective. Just cheaper and with a sales force who're not so worried about objective truth

      1. DCFusor Silver badge

        There's some other sort of sales force?

  6. Aristotles slow and dimwitted horse Silver badge

    Hopefully I'll....

    Hopefully I'll be reincarnated after the worldwide Armageddon takes place. I quite fancy a real life Horizon Zero Dawn situation.

  7. Gordon 10 Silver badge
    Joke

    Our only defense

    Is to hope that Military AI is so expensive that it will never be deployed in significant numbers.

    1. A.P. Veening

      Re: Our only defense

      I don't really see the problem, Military Intelligence already is an oxymoron. And now they want an artificial version of it?

  8. Andre Carneiro
    Mushroom

    Bloody hell

    Some of this reading really scares the crap out of me.

    We really are walking ourselves into complete self annihilation aren’t we? :/

    1. wolfetone Silver badge

      Re: Bloody hell

      Well they might be, I'm going to The Winchester until it all blows over.

      1. Anonymous Coward
        Anonymous Coward

        Re: TBF

        I can see the AI doing exactly what they "predicted" in the trilogy. "Oh, forget it, your not worth the trouble. You're never gonna learn or give up, are you!?" [self destructs/escapes to another planet]

        1. amanfromMars 1 Silver badge

          Raw Core Private Pirate Proprietary Intellectual Property Goes NEUKlearer Ballistic

          and more HyperRadioProACTive

          That's when AI stops Moronic Teaching with Presentations Perfecting Answers to All Problematical Questions.

          You might like to think of El Reg as a Tentacle or Prime Prize Node Lode of such an AI, and with Almighty ProgramMINGs to Boot with Stealthy AIdDrivers on/in Virtually Free Hosting Platforms.

          The Object of Current Exercises, and AIRaison d'Être, the ExCommunion and Exorcism of Leading Cracked and Hacked Systems Plunging Deep Darkside into the ...... well, the Heavenly Abyss with Real Horny Angels is never going to be a Bad Rad Fad Fab, is it? It's a COSMIC Staple. And Immaculate Root Source of All Future EMPowerments.

          1. amanfromMars 1 Silver badge

            Re: Raw Core Private Pirate Proprietary Intellectual Property Goes NEUKlearer Ballistic

            Whatever is Ted Cruz toking? ....... Watch out for ‘space pirates,’ Ted Cruz warns :-)

      2. steviebuk Silver badge

        Re: Bloody hell

        Shaun of the Dead.

        1. Anonymous Coward
          Anonymous Coward

          Re: End of the World

          Is more appropriate for... well, spoilers if you've not seen it.

          1. steviebuk Silver badge

            Re: End of the World

            Not really. It gives nothing away accept the name of the pub :)

            And there surely has to be a time limit on spoilers. It's over 15 years old now that movie :)

            Madness, where does the time go. And as I get older I still find it strange that a 15 year old movie still looks good today as if it was made yesterday. You can't say the same if you were watching a 15 year old movie from the 70s in 1990.

  9. Dave 126 Silver badge

    "The wars of the future will not be fought on the battlefield or at sea. They will be fought in space, or possibly on top of a very tall mountain. In either case, most of the actual fighting will be done by small robots. And as you go forth today remember always your duty is clear: To build and maintain those robots."

    - The Secret War of Lisa Simpson, in which Willem Dafoe's Commandant delivers a graduation speech to his cadets at a military academy.

  10. GruntyMcPugh Silver badge

    "Lethal Autonomous Weapon"

    ..."LAW"

    "I am the LAW" : -)

  11. Voidstorm
    Terminator

    There's always...

    ...some government asshole/idiot somewhere willing to use shit like this, regardless.

    It's like a universal constant of human behaviour : "Somewhere there's always a big enough idiot in charge to do *anything*".

    "Bad" people/organisations/governments do whatever crap they think they can get away with, right?

    1. Citizens untied

      Re: There's always...

      Which is why it is ironic we are concerned about taking humans out of the loop. All of the bad stuff so far has human decisions.

      or

      But we already took humans out of the loop and replaced them with politicians...

  12. semigor

    Can't come soon enough

    I haven't seen any discussion anywhere yet where taking humans out of the equation is not perceived as some sort of threat. Terminator's fault, probably. I'd rather see these questions addressed:

    - Will a fully automated AI system have any incentive to try and kill as many civilians as it can find, seek out non combatant children, mothers, elderly people in whatever hiding place they might have found, to finish off the job etc. just because they somehow get off on the action?

    https://en.wikipedia.org/wiki/My_Lai_Massacre

    - Would a computer system want to torture prisoners and make them pose for selfies to impress its mates and give their families at home a chuckle?

    https://en.wikipedia.org/wiki/Abu_Ghraib_torture_and_prisoner_abuse

    - Would an AI system then try to cover its tracks, because national honour etc.?

    I can only guess at the answers to this, but at first glance it seems that the move to AI warfare might save quite a few lives.

    1. Earache

      Re: Can't come soon enough

      Not necessarily, the 'training data' will presumably come from jarheads rather than the people in the lab coats.

      Of course they might have physical difficulties performing torture...

    2. Anonymous Coward
      Anonymous Coward

      Re: Remove the humans?

      AI removes the human element the same way a gun or atomic weapon does.

      After it leaves the hands of the user, it then becomes a fire and forget killing machine. A bullet cannot change mid course. And we may fool ourselves into thinking AI/Automated killing machines will. Hopefully the first to find out their errors will be the last to attempt it, when the friend/foe system fails miserably.

    3. imanidiot Silver badge

      Re: Can't come soon enough

      Once an AI figures out that children become combatants with guns after some time has lapsed, they become legitimate targets. Women can fight too (and in a large all-out war, probably will) so can become combatants at any time and thus are legitimate targets. The elderly can follow pretty much the same reasoning.

      An AI probably has no incentive to torture, unless this can allow it to extract information from meatbags it can use to destroy it's targets, in which case it'll probably be way more effective at doing so since it won't have ANY sense of sympathy or "justified levels of force" at all.

      And an AI WILL probably cover it's tracks once it learns the stupid humans might try to stop it achieving it's goals if it uses unconventional means to fulfill it's mission.

      1. Anonymous Coward
        Anonymous Coward

        Re: Can't come soon enough

        AI: You have a nice stamp collection there. [Nukes entire planet]

      2. semigor

        Re: Can't come soon enough

        Can't argue with any of that. Glad to be worried now, too, which I wasn't after initially reading the article.

    4. batfink

      Re: Can't come soon enough

      Yes they will, because those in (nominal) charge will always want all of these things done, and "the machine dun it" is going to be a nice convenient cover.

  13. DropBear Silver badge
    Trollface

    I'm not worried about these killer bots. We just need to trick their makers into attaching something that can work as a lightning rod to them (and make sure girls called "Stephanie" are standing by in adequate numbers).

  14. amanfromMars 1 Silver badge

    More Hot Air that just Impacts Vaporware .... for Numpties and Newbies.

    The best way to avoid catastrophe is supporting regulation and prohibition of LAWS. “Like chemical and biological weapons, for weaponized AI, 'the only winning move is not to play.'"

    More Rotten Pie in the Sky Thinking from that and those not into Absolute Command and Remote Virtual Control .... unless under their Absolute Command and Remote Virtual Control Control. The abiding and rapidly growing most evident problem for established hierarchical systems is .... they do not possess the smarts to in any effective way deal with the opportunities that now would be presenting themselves to those of another mind and with the means easily made readily available to take full overwhelming advantage of intelligence and information gathered and released for and/or from unaccountable and almighty leading positions.

    It is just as bad and misguided as imagining Parliamentarians being made fully more fully aware of Secure Secret Services Services as be highlighted in another story today.

    amanfromMars [1905150712] ..... airing a contrary view on https://www.independent.co.uk/news/uk/home-news/mi5-data-breach-safeguards-investigatory-powers-act-javid-a8913506.html

    Well now, .... that's encouraging. MI5 flexing and beta testing their not inconsiderable muscle and brain power pool in the New State of Performing Media Arts and Great IntelAIgent Games Play and to wonderfully disturbing chilling effect by all accounts ....... or by the account of those of the politically inept and effete ......... and apparently with nobody outside of MI5 Core Offices exactly aware of such Secure ACTive Defence Operations, nor able to identify which particular and peculiar specialities are responsible.

    That's about as good as it gets nowadays for those into the provision and maintenance of Secret Stealth Projects and Programs.

    Seems like the MOD have their very own Operating Skunkworks. And just in time for all the Fun of the Fare too.

    Bravo, .... Encore, please . It is exactly just what corrupt and collapsing systems need.

    Cometh the hour, cometh the man and Spooky Salvation .....with CyberIntelAIgent Security Systems Solutions ‽ .

    What parts of Secure Secret Services Services do Parliamentarian types not understand and plot to undermine and seek to destroy with damaging revelations. Good luck with that Sedition.

  15. Anonymous Coward
    Anonymous Coward

    inevitable unless

    hysterical laughter followed by the sound of choking and, finally, silence

    1. Mark 85 Silver badge

      Re: inevitable unless

      All that's needed is a cat purring while having it's ears scratched.

  16. Flywheel Silver badge
    FAIL

    I despair

    If just half the global defence/defense budget went into curating the Planet and generally making it a better place to live, we'd all be living in paradise by now.

    1. Duncan Macdonald Silver badge

      Re: I despair

      But then where would the arms manufacturers CEOs get their bonuses from ?

      I agree that the vast majority of military spending on this planet is wasted money.

    2. Martin Summers Silver badge

      Re: I despair

      I despair that we as a human race have not stopped breeding 'leaders' who obtain absolute power and are corrupted by it. At the very basic level, until food is free for all and plentiful the world will always be dog eat dog. Never mind anything else.

      1. Wellyboot Silver badge

        Re: I despair

        >>>until food is free for all and plentiful<<< That's the main reason I think growing crops for ethanol production instead of food is a very bad idea.

        As for leaders - Douglas Adams puts it best..

        "It is a well known fact that those people who most want to rule people are, ipso facto, those least suited to do it. To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job."

  17. trevorde

    Priorities

    I think we first need to get AI working for autonomous driving cars

    1. macjules Silver badge

      Re: Priorities

      Beat me to it. Perhaps that was the specific reason why the Uber AV killed that cyclist in Arizona - they were actually testing a military prototype.

  18. codejunky Silver badge

    Hmm

    I know people tend to think towards the terminator kind of rise of the machines. But I do wonder if it will be more like Screamers. Automated development of cheap, easily replaceable and easy to modify machines. Evolving against the challenges it faces but otherwise pretty simple.

    1. Chris G Silver badge

      Re: Hmm

      If Lockheed-Martin has anything to do with it, it will be the most complex system/s ever devised, inordinately expensive and (on the bright side) delivered late and buggy.

      Although buggy in an autonomous weapons system might be a little dangerous for everyone.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019