back to article UK publishes Laws of Robotics for self-driving cars

The United Kingdom has published a set of “Key principles of vehicle cyber security for connected and automated vehicles” outlining how auto-makers need to behave if they want computerised cars to hit Blighty's byways and highways. Penned by the UK's Department for Transport, with help from the Centre for the Protection of …

Page:

  1. TheElder

    This includes sensor jamming or spoofing.”

    Regarding the laser jamming previously posted here: What should it do when it is suddenly blind? Jam on the brakes? Pedal to the metal? Turn off the road (bridge, overpass)?

    Possible Jammer?

    1. Lysenko Silver badge

      Re: What should it do when it is suddenly blind?

      Presumably, the same thing you (or more precisely, a panel of expert drivers) would do if they suddenly lost vision. Personally, that would mean a controlled emergency stop and hitting the hazards.

      Cars are already controlled by wetware computers with sensor and control systems that can fail in various ways. The appropriate responses are therefore a solved problem (insofar as a "solution" is actually possible).

      1. Lee D Silver badge

        Re: What should it do when it is suddenly blind?

        So, er... how waterproof are these laser sensors? Are you going to hit a puddle and then suddenly be put into an emergency stop?

      2. Primus Secundus Tertius Silver badge

        Re: What should it do when it is suddenly blind?

        @Lysenko

        A long time ago, when diesel lorries used to emit thick black exhaust fumes, I was driving one night. I could see a lorry coming the other way, lit up like a Xmas tree. What I did not see, because it was night, was its thick black exhaust.

        As we passed, suddenly I could see nothing. I was doing about 50mph, totally blinded. I started braking, and after a second or so emerged from the smoke without incident. Angry, mind you.

        1. The First Dave

          Re: What should it do when it is suddenly blind?

          Indeed - the correct response to loss of vision is _gentle_ braking (to a standstill if necessary). This assumes that one was driving within the limits of vision beforehand of course, but if you drive into a bank of fog and do an emergency stop, someone WILL run into the back of you.

        2. Lysenko Silver badge

          Re: What should it do when it is suddenly blind?

          @Primus

          That still happens when an HGV kicks up spray on the motorway: you have temporary visibility degradation and you know what caused it. A computer may not be able to determine that so it would have to respond as a human would to actual loss of vision.

          For example, if everything suddenly goes black for no apparent reason then there is a strong possibility of neural hypoxia (e.g. catastrophic blood pressure loss) and you might be looking at imminent cascade failure of all cognitive functions. Even if you can't feel other symptoms, you know your sensors are compromised and there's no way to know how much time you have before motor function failure. Hence: emergency stop.

  2. Haku

    Fucking with the 'robots'.

    I read a comment recently about drivers who have been screwing with the autopilot feature of Teslas; whilst on freeways people purposely swerving into the lane of a Tesla driver so their car has to react, ie automatically braking or changing lanes.

    I did try and find video evidence of this happening, but all I could really find was YouTube videos of how the autopilot has avoided crashes due to the incumbency of other drivers (often rather reasurring evidence of how computers will spot something a human driver misses) but also the occasional piece of footage of the car screwing up and swerving into oncominrg traffic, causing the driver to take back control of the steering wheel, indicating the technology still has a long way to go before full automation on the roads can be completely trusted.

    It leaves me wondering what the future of driving will be like when the ratio of computer to human drivers gets close to 1:1.

    In the case of human drivers 'trolling' the computer drivers, swerving into the path of them etc., at least those computer driven vehicles will have dashcam evidence of the himan drivers acting like arseholes, because who would design a computer driven vehicle without cameras recording everything that's going on for in the event of something going wrong.

    1. Baldrickk Silver badge

      Re: Fucking with the 'robots'.

      And that's what 3.4 is for.

    2. Seajay#

      Re: Fucking with the 'robots'.

      This is not a feature of self-driving cars, it's a feature of dickheads. What do you think happens with human driven cars when someone "trolls" them by swerving in front of them? I bet they behave a lot worse than the autonomous cars.

      Also "trolling" is the wrong word here, "attempted murder" is better.

      1. justAnITGuy

        Re: Fucking with the 'robots'.

        Fancy blaming the robots for humans fucking with them! Some on here evidently never having seen or heard of "Cash For Crash" where dickhead humans deliberately cause RTA with other non-dickead humans to make fraudulent insurance claims. Replace non-dickead human with a future AV and the chances of the CFC driver succeeding become more remote.

    3. Version 1.0 Silver badge

      Re: Fucking with the 'robots'.

      Driving around the Welsh roads on vacation recently, I was wondering how a Telsa would deal with them? Has anyone tried driving two Telsa's down a Welsh road and seeing how they handle a meeting?

      Driving is a skill - if you don't use it, you lose it.

    4. Aitor 1 Silver badge

      Re: Fucking with the 'robots'.

      Camera, and police, that is the future.

    5. TheElder

      Re: Fucking with the 'robots' (and the cameras).

      In the somewhat distant past I installed a special filter over my vehicle identification tag.

      Privacy filter

      Can be seen from directly behind but not to the side at all. When I showed it to the RCMP I worked with they were not amused. They thanked me. I also instructed them on various laser jammers as well. I have always been a teacher.

  3. This post has been deleted by its author

  4. DougS Silver badge

    "Ensure systems are secure over their lifetime"

    That's a laudable goal, but who owns support for cars after their OEM goes bankrupt? That's bound to happen, with all the traditional carmakers getting into the game, plus various new entrants. A lot of carmakers will go bankrupt, because people aren't going to be buying cars as many cars as they do now once ownership becomes the exception rather than the rule.

    Not only do vendors need to commit to lifetime support, they need to purchase insurance against the possibility they can no longer do so to fund someone else taking over. If Volvo for example went bankrupt next year, a Volvo you buy today is still just fine. There are people who will be able to service it for you. But an autonomous car is useless without updates that provide necessary security and fix potentially deadly bugs that may be found in their code many years after sale. No one would insure such a car, so it would become an extremely expensive paperweight with no one left to maintain its software.

    1. Lysenko Silver badge

      Re: "Ensure systems are secure over their lifetime"

      Insist they escrow the source code and open source it in the event of liquidation. Extra points for open sourcing from the start. I'm betting there will be a lot of Linux and other GPL/MIT/BSD etc. code floating around in many of these systems anyway.

    2. Pascal Monett Silver badge

      Re: "Ensure systems are secure over their lifetime"

      It is an issue for all things computer-related, but car makers never actually disappear. A brand may go bankrupt, but its assets are generally gobbled up by someone else. Volvo Cars, for example, belongs to Ford. That means that the chances for a car maker to just disappear are much smaller than for the makers of furry bears for children.

      That said, threats do not disappear and any given model of car is likely to have a road presence that goes largely beyond the decade, meaning that the manufacturer will be responsible for keeping the updates available for that model for however long there are still cars of that model running. In other words, it would be very badly received if a car maker announces that support for a given model will be retired because it stopped making it a decade ago.

      That is something the manufacturers will have to bear in mind and it would be good for them to elaborate a global framework that they implement in all their cars, so as to not go foul of that kind of issue. Which is an obvious solution and I'm sure they've already thought of it as well.

      1. Thesheep

        Re: "Ensure systems are secure over their lifetime"

        <pedant>

        Volvo aren't owned by Ford anymore. They are owned by Zhejiang Geeley Holding Group.

        </pedant>

      2. Chris G Silver badge

        Re: "Ensure systems are secure over their lifetime"

        This presents a relatively good reason for developing an industry wide standard for autonomous driving systems, rather than a diiferent set of code for each manufacturer and/or model of car.

      3. 's water music Silver badge

        Re: "Ensure systems are secure over their lifetime"

        ... any given model of car is likely to have a road presence that goes largely beyond the decade, meaning that the manufacturer will be responsible for keeping the updates available for that model for however long there are still cars of that model running. In other words, it would be very badly received if a car maker announces that support for a given model will be retired because it stopped making it a decade ago

        Well I suppose they could announce a service life before the model goes on sale. Fleet operators and contract hire providers wouldn't care if they went into the deal with open eyes. There would come a point when it would be cheaper to compensate the remaining owners. Again not really a problem if the cutoff date and terms were declared prior to first sale.

        1. jmch Silver badge

          Re: "Ensure systems are secure over their lifetime"

          It used to be the case that any car whatever its age could be maintained by any competent mechanic. Nowadays any car model from around 2010 (earlier for some upmarket brands) has so much electronics that it needs to be serviced at the licensed dealer or else hacked into. This makes servicing more difficult and expensive, which means that the point where it makes more financial sense to buy a new car rather than service the old one comes earlier and earlier in the car's lifetime. (This is probably at least partially by design)

          I don't think it will be uncommon in the future for manufacturers to commit to say 15 or 20 years of guaranteed availability of servicing, as long as they publish their software interfaces to allow independent mechanics to work on older models without hindrance.

          Official dealers would themselves probably rather only have to service newer models, and independent mechanics still have a steady (albeit small) stream of customers. Win-win?

    3. nathanm

      Re: "Ensure systems are secure over their lifetime"

      Insurance seems a pretty good option. That, combined with the board being accountable for security. So even if the company goes bankrupt, if they didn't take out adequate insurance when solvent, someone can still be brought to task.

      1. Version 1.0 Silver badge

        Re: "Ensure systems are secure over their lifetime"

        "... someone can still be brought to task." - ROTFLMAO ... like when did that ever happen?

        Corporation goes belly up, Insurance is found to be missing, the bosses say that it's not their fault, blame the accountant, the accountant says that renewal was automatic, its not their fault and everyone moved to new companies with a big fat bonus.

        1. DougS Silver badge

          Re: "Ensure systems are secure over their lifetime"

          Insurance companies that will be handling the liability insurance (i.e. if the car kills someone) can require proof of adequate "maintenance insurance" as a condition of covering the car, whether or not the automaker or the owner is responsible for paying for that insurance. Or maybe just roll the cost for that into the liability insurance - since it is obviously in the interest of the company underwriting the liability policy to insurance that security fixes are made in a timely manner!

    4. I am the liquor

      Re: "Ensure systems are secure over their lifetime"

      The big omission from principle 6 is the failure to define "lifetime". What's the betting the car firms will interpret it as "warranty period."

    5. Mark 85 Silver badge

      Re: "Ensure systems are secure over their lifetime"

      I think everything will planned as it is now... "planned obsolescence". It will just be a matter of juggling some numbers to shorten the lifetime. Presumably, in the future, there won't be any 10 year old cars (or any of 'classics') on the roads not just by design but by edict.

    6. Anonymous Coward
      Anonymous Coward

      Re: "Ensure systems are secure over their lifetime"

      Meanwhile, just remind me what other (non-military) product on the planet complies with this sentiment?

      TVs don't, Mobile phones don't, PCs don't.

      If they can't do it for what are essentially IT products, they certainly won't be doing it for cars. Unless they define the life as 5 years maximum.

  5. conscience

    I'm still not sold on the idea of autonomous cars, but making the board responsible for security of the hardware and software is a great idea.

    Maybe making the board personally accountable should be extended to all computer based products, there are many things are crying out for some security update love.

    1. VinceH Silver badge

      "I'm still not sold on the idea of autonomous cars, but making the board responsible for security of the hardware and software is a great idea."

      Well, yes, but don't forget the sub-principle quoted in the article (my emphasis): “Personal accountability is held at the board level for product and system security (physical, personnel and cyber) and delegated appropriately and clearly throughout the organisation.”

      I read that as "The board is accountable, but when things go pear-shaped, they can delegate that accountability to an appropriate scapegoat."

      1. Mark 85 Silver badge

        I would like to see someone in corporate with the job title of "Corporate Scapegoat". I offered myself for this position many years ago but it was declined. When the company went to hell in a handbasket, the board scrambled to find someone to blame...

  6. big_D Silver badge

    But but but...

    The communication must be secure? But the government wants to ban encrypted communications!

    1. Headley_Grange Silver badge

      Re: But but but...

      @Big_D -- When you pay by credit card in a shop the communication link is secure but the information is available at the bank for when it's needed; e.g. to send you a statement. The government doesn't want to ban encrypted information links, it wants to make sure that the information is getatable at some place- either the user's phone or the supplier's servers.

      1. Aitor 1 Silver badge

        Re: But but but...

        errr, no, the government wants to break encryption. They want to be able to access your communications when they end in places they cannot access, and also communications that start and end outside the UK (same for the the US).

    2. Smooth Newt Silver badge
      Stop

      Re: But but but...

      The communication must be secure? But the government wants to ban encrypted communications!

      Fortunately (for the Government), the whole document is so packed full of weasel words that it is pretty much meaningless. The document looks designed to give you a warm feeling, but not actually do you any good, a bit like when you piss down your leg. What do you think "sufficiently secure", "managed appropriately", "where possible" etc actually mean below. This is the entire text of the Principle 7!

      "Principle 7 - the storage and transmission of data is secure and can be controlled

      Principle 7.1

      Data must be sufficiently secure (confidentiality and integrity) when stored and transmitted so that only the intended recipient or system functions are able to receive and / or access it. Incoming communications are treated as unsecure until validated.

      Principle 7.2

      Personally identifiable data must be managed appropriately.

      This includes:

      what is stored (both on and off the ITS / CAV system)

      what is transmitted

      how it is used

      the control the data owner has over these processes

      Where possible, data that is sent to other systems is sanitised.

      Principle 7.3

      Users are able to delete sensitive data held on systems and connected systems."

      1. Headley_Grange Silver badge

        Re: But but but...

        @Smooth - is it a problem that the document is woolly? If UKGOV had tried to define anything (specs, protocols, etc) then it would have been slated for shackling industry, outdated after about 20 minutes and possibly also acted as a hacker's manual by helping to define attack vectors.

        The woolliness might allow companies to wriggle out of stuff, but it's a feature certainly of English law; ALARP, best endeavours, the man on the Clapham omnibus etc. abound and in my experience companies take their legal responsibilities in these woolly areas just as seriously as in better defined ones. The supposedly tight, technical specs around emissions testing and compliance didn't prevent car companies trying to find ways around it and any eventual prosecutions in Europe will not simply be about not meeting the specs (which is sort of proven), but about proving an intent to defraud.

        1. Smooth Newt Silver badge
          Happy

          Re: But but but...

          The supposedly tight, technical specs around emissions testing and compliance didn't prevent car companies trying to find ways around it and any eventual prosecutions in Europe will not simply be about not meeting the specs (which is sort of proven), but about proving an intent to defraud.

          But they only found way around it by, allegedly, breaking the law. If the regulations had vaguely said that "nitrogen dioxide emissions should be managed appropriately" then the law would be so vague that any prosecutions would be impossible.

          The principles need to reference subsidiary standards so that we know what, e.g., "sufficiently secure" means. As it stands, "Sufficiently secure" means absolutely nothing, since "sufficiently" is just a matter of opinion, and what happens if a manufacturer thinks, e.g. DES encryption, is "sufficiently secure"? Standards mean you have to justify what you are going to do in advance if you want to influence the standard, rather than "screw it, let's do it" and only worry about the consequences when someone gets hurt.

      2. Anonymous Coward
        Anonymous Coward

        Re: But but but...

        In other words, more bad law. (At least it's not secret).

        That must be a first :/

  7. Teiwaz Silver badge

    Security

    ensure systems are secure over their lifetime;

    And how does our esteemed UK gov think that is going to be possible, given that they also want to ban encryption???

    Ah yes, this is from Transport, not the frothing insane leg of the establishment.

  8. moiety

    Sounds mostly sensible. This bit had me worried:

    "The storage and transmission of data is secure and can be controlled"

    ...because it doesn't say what "controlled" means. Zooming in a bit via the original document (link) gives you:

    ==============8<===================

    Principle 7.1

    Data must be sufficiently secure (confidentiality and integrity) when stored and transmitted so that only the intended recipient or system functions are able to receive and / or access it. Incoming communications are treated as unsecure until validated.

    Principle 7.2

    Personally identifiable data must be managed appropriately.

    This includes:

    > what is stored (both on and off the ITS / CAV system)

    >what is transmitted

    >how it is used

    >the control the data owner has over these processes

    Where possible, data that is sent to other systems is sanitised.

    Principle 7.3

    Users are able to delete sensitive data held on systems and connected systems.

    ==============8<===================

    ...which is a bit better, but still fails to nail down who the parties are. Users, presumably are going to be either the owners or users of the cars (might be the same person; might no in the case of fleets). "Data owner" is a bit slippery too....the manufacturers are going to own the software running the car and updates; but it's not clearly specified who owns journey data.

    Also it's a certainty that the government/spooks/police are going to want to deal themselves in at some point in this process and this document doesn't seem to cover even the possibility of that.

    1. steelpillow Silver badge

      Data owners, ooer

      @moiety: The "data owner" is a well-defined concept in infosec and refers to the person about whose activity or business the data relates. For example journey information belongs to the driver and any passengers. Hire cars must begin to pose headaches in this respect: who owns the "black box" recorder data, the vehicle owner or the driver, if it hasn't already been sold to Google by the manufacturer. :(

  9. David Roberts Silver badge

    It is now a computer/phone

    So you need the same precautions when you sell it.

  10. smudge Silver badge
    Thumb Up

    ISO27001/27002

    Looks like an interpretation of ISO27001/27002 for a specific area of application.

    Which is fine - when I was working it was always useful as a good starting-off point, especially with the recent(ish) updates which put more emphasis on the supply chain.

  11. Peter Prof Fox

    Fail

    Just suppose I was ever driving such a thing and, say the robot runs a red light. To defend myself I need access to the code and data to prove the 'blindspots' in the system are the robot's fault and not mine. Remember the hassle people had getting access to speed camera code? This will be a lot worse.

    1. earl grey Silver badge
      Stop

      Re: Fail

      You don't understand. When all the meatsacks are gone or not allowed to drive, the automobots won't have a need for red lights and so that technology will simply go away.

  12. Primus Secundus Tertius Silver badge

    Good software design needed

    Underlying all the good intentions listed in the report is a requirement that the software design be first class. This means looking ahead during the design stage at all the things that might go wrong, and dealing with them without letting the system crash.

    This is much more effort than designing only for when things go right, and is discouraged by most managers. So external validation will be necessary.

    At a minimum, all software should be put through an official driving test.

  13. steelpillow Silver badge

    9 out of 10

    Oops, 6.4 has blown it: "Software adopts open design practices and peer reviewed code is used where possible. Source code is able to be shared where appropriate." So a Bad Bunny can cut rogue code, hide it in a "commercially sensitive" blob and refuse to let anybody else vet it. No no nooo!

    It should read, "Software adopts open design practices and all code must be peer reviewed within one month of release. Source code is to be made publicly available and reviews of all third-party bug reports to be published alongside them."

    That "one month" allows grace for urgent security patches.

  14. steelpillow Silver badge

    The old chestnut

    Missing is, "Any system containing critical safety functionality must never initiate a communications session with a system which does not contain such functionality."

    It's the old "the server may not initiate a client session" rule, designed to contain any service compromise, which so many system designers struggle to come to terms with.

  15. Wupspups
    Joke

    Organisational security is owned, governed and promoted at board level;

    Fat chance of that when the board is beholden to the shareholders!

  16. TheElder

    Driving safely

    If they start putting a lot of these things on the road I will need to change my mode of transportation.

    Safe Driving

  17. Anonymous Coward
    Anonymous Coward

    Rules? We don't need no stinkin' rules!

    Since when have government rules and guidelines been any good?

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019