back to article Revealed: How Nvidia's 'backseat driver' AI learned to read lips

When Nvidia popped the bonnet on its Co-Pilot "backseat driver" AI at this year’s Consumer Electronics Show, most onlookers were struck by its ability to lip-read while tracking CES-going "motorists'" actions within the "car". A slide taken at CES shows the Co-Pilot AI assistant performing four features: facial recognition, …

  1. Jess--

    I predict that the system will learn a lot of new swear words when this is used in live traffic

    1. m0rt

      "I'm sorry, Dave, I am not physically equipped to do that..."

    2. FuzzyWuzzys
      Happy

      Loud music anyone?

      Especially for those of us who enjoy listening to niche, aggressive music while driving. I can imagine the fun I'd have screaming along to Cannibal Corpse or Napalm Death and the Nvidia kit suddenly decides I'm in danger and pulls the car over into the hard shoulder!

      1. Hans 1
        Thumb Up

        Re: Loud music anyone?

        >Cannibal Corpse

        Hammer Smashed Face - EPIC

        I wonder when the system will learn gestures as well ...

        https://www.youtube.com/watch?v=OIr3YEioMWw

    3. joed

      Yep, curse-words are easy to read.

      Now, it's likely that the whole process requires network connectivity and this is good enough reason not to ever use it. I like to keep it to myself without wearing a full face helmet. If Nvidia's CEO likes a snitch in his car it's his choice.

  2. Dabooka

    Old news

    My commute sees the majority of cars seemingly being driven by some kind of artificial intelligence.

    1. tony2heads

      Re: Old news

      I have the opposite phenomenon; they seem to be driven by non-artificial stupidity.

      1. Jess--

        Re: Old news

        He said it appeared to be driven by some kind of artificial intelligence.

        He didn't say that it was switched on (or that it was the intelligence of an amoeba)

    2. 2460 Something

      Re: Old news

      As much as I love driving, I would still love one of these for longer journeys, I'm thinking I, Robot style jumping between manual and automation. Just need swivel seats in the front and it's good to go! Another 10-15 years and they will probably be mainstream as opposed to niche.

  3. Anonymous Coward
    Anonymous Coward

    Facial recognition to start cars...

    ... is as useful as smart guns with fingerprints readers. Good luck when you're incapacitated, and someone could save you driving your car...

    1. DaveyDaveDave

      Re: Facial recognition to start cars...

      Are you some kind of spy or gangland boss? I'm really struggling to imagine any other moderately likely scenarios where this would be helpful. If I'm incapacitated, I would very much prefer someone to call me an ambulance, than drive me anywhere. If I'm in immediate danger, then being helped to walk (or dragged if absolutely necessary) away from that danger will be fine.

      If I'm missing something obvious, and this would actually be necessary, I imagine there would be some kind of emergency override, like all phones have for calling the emergency services.

      1. BebopWeBop
        Holmes

        Re: Facial recognition to start cars...

        A gangland boss would know you can just lop the head off the 'should be' driver to gain access - and would be prepared to do it

  4. sorry, what?
    Black Helicopters

    Local processing is a great thing

    I like that they are using local processing to collect information from the driver (and other vehicle occupants?) unlike lots of other solutions that feed the raw stuff to a server (or two) for processing.

    Of course, I'm guessing the software will then squirt all the extracted information straight to our digital overlords. Which, for me, would be the deal breaker.

  5. Lotaresco

    Cool

    Can I get it coupled with a laser to burn what I'm saying on the forehead of the oncoming idiot?

  6. MrT

    Let's hope it's not too connected...

    Backseat passengers mouthing "Tesla, please get me an Amazon Echo ... Yes", or just using sign language...

  7. Anonymous South African Coward Bronze badge
    Trollface

    Wonder what it'll do if the driver throws a hissy fit/road rage tantrum?

  8. AceRimmer1980
    Paris Hilton

    I saw you in the dealership, Dave

    You were asking the salesman how much you could get if you traded me in..

  9. Nimby
    Thumb Up

    Recognize this!

    The important missing feature is the ability of the driver to program phrase and gesture recognition events into performing tasks and functions of their desire. I foresee my car having an awful lot of events triggering the horn. Soon *honk* will become the new universal adjective.

    The facial recognition in place of a key is an interesting idea. That way you can tell your brother-in-law that he can borrow the car whenever he wants, safe in knowing that he never can. Though I do sort of wonder how someone can get into a locked car without any key if the camera faces the driver's seat. Lots of faces smashed into windows, hoping to catch the edge of the camera's field of view? Or just the good ol' fob still needed to be carried everywhere like the key that it is.

    Worse, on the opposite side, facial recognition as a key could be a serious security hole. Just print off a picture of someone from their social media of choice and away you go, in their car!

    1. Antron Argaiv Silver badge

      Re: Recognize this!

      Your icon, though almost accurate, seems to be using the wrong digit.

  10. Anonymous Coward
    Anonymous Coward

    Maybe El Reg's picture editor is too young...

    ...to remember this iconic image?

    https://i.ytimg.com/vi/1s-PiIbzbhw/maxresdefault.jpg

    1. Francis Boyle Silver badge

      I was a bit disappointed

      that El Reg couldn't manage a Hal reference but the Kipper loving one's post upthread more than made up for that.

  11. Anonymous Coward
    Anonymous Coward

    "an impressive accuracy rate of 93.4 per cent"

    Impressive to whom?

    To technologists who can imagine what tracking and interpreting facial features in real time involves, yes it's bloody amazing. But if you were sitting a language exam and got one in fifteen words wrong, you wouldn't score very highly.

  12. Oengus

    New accessory

    I think I will start selling a new accessory for these type of cars. It will be a small black patch to stick over the camera lens. Now read my lips...

    1. joed

      Re: New accessory

      Zuck has some to share. Just click the like button.

  13. joe bixflics

    It better learn to recognize the phrase, 'Oh my God'.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like