back to article Alexa heard what you did last summer – and she knows what that was, too: AI recognizes activities from sound

Boffins have devised a way to make eavesdropping smartwatches, computers, mobile devices, and speakers with endearing names like Alexa better aware of what's going on around them. In a paper to be presented today at the ACM Symposium on User Interface Software and Technology (UIST) in Berlin, Germany, computer scientists …

  1. Doctor Syntax Silver badge
    Mushroom

    Just because you can do a thing it doesn't mean you should.

    Icon: tape over the mic might not be enough.

  2. sms123

    Alexa hears shouting and a dropped knife.

    Says: Would you like me order you a shovel, lye, and a body bag from Amazon Prime?

  3. Francis Boyle Silver badge

    Nah

    They trained it on movie effects. Your victim would have to do the Wilhelm scream for that to work.

  4. jmch Silver badge
    Facepalm

    "Just because you can do a thing it doesn't mean you should."

    Spot on. And anyway what's so difficult about having a button on the wall or on your smartwatch that activates the listening, which is otherwise not active? Is that too much of an effort ??

  5. The Man Who Fell To Earth Silver badge
    WTF?

    Great - IoT peeping Tom Audio porn

    What could possibly go wrong?

  6. Pen-y-gors Silver badge

    Confusion?

    But it could be fun trying to confuse Alexa.

    Many moons ago a friend got a BBC sound effects tape and had fun with the message on his ansaphone - sounds of police sirens, helicopters and shooting in the background, to a gentle voice-over of "I'm afraid I can't get to the phone just at the moment, please leave a message"

    Would Alexa call the plods if it heard shooting?

  7. Version 1.0 Silver badge

    Re: Confusion?

    Play Big Youth's "Screaming Target" and see what happens...

  8. RegGuy1
    Devil

    What's so difficult about having a button on the wall ...?

    Nothing. It's just that it's on their wall.

  9. tfewster Silver badge
    Facepalm

    Forgive my ignorance of the state of the art - But can it distinguish "live" sounds from a TV or radio?

    (I read the article twice, and all the comments, and no-one seems to have addressed this elephant in the room)

    On the other hand, this could be incredibly useful for relaying the wife's softly-spoken orders - usually issued when I'm in another room/beside a boiling kettle or running tap/wearing headphones.

  10. John Brown (no body) Silver badge

    Just because you can do a thing it doesn't mean you should."

    And for that matter, who do something you don't even need to do?

    "The researchers suggest their system could be used, for example, to send a notification when a laundry load finished."

    Surely your "smart" washing machine will email you when it's finished.

  11. onefang

    "Forgive my ignorance of the state of the art - But can it distinguish "live" sounds from a TV or radio?"

    The article did say they trained it with sound effects from Hollywood, so my guess is no. Or alternatively yes, except the sounds from the TV will be taken as being "real", but the "live" sounds are obviously fake.

  12. Korev Silver badge
    Joke

    >And anyway what's so difficult about having a button on the wall

    You could even.... gasp... turn your lights on and off with one :o

  13. GruntyMcPugh Silver badge

    @onefang: "sound effects from Hollywood"

    So I'm going to leave 'that scene' from 'When Harry Met Sally' on a loop, and see if I get any interesting shopping suggestions. I'm guessing lube.

  14. HandleAlreadyTaken

    Yeah

    That's not creepy at all...

  15. Anonymous Coward
    Anonymous Coward

    Re: Yeah

    Quite, and the researchers saying 'this should all be doable locally as people don't really want to be spied on' is like saying 'Lions don't really want to hurt people' - but they have to eat, right?

  16. Outer mongolian custard monster from outer space (honest)

    Re: Yeah

    "Quite, and the researchers saying 'this should all be doable locally as people don't really want to be spied on' is like saying 'Lions don't really want to hurt people' - but they have to eat, right?"

    Sure, but put away your paranoia for a instant and remember this is CMU saying this, as in the people that developed CMU Sphinx, and pocket sphinx. Which today is about the best bet for a local only speech recognition system as your able to compile/build at home. For that they should be at least given a small amount of the benefit of the doubt that they were being sincere.

    I should know, I've been making my own smart speaker that is local network only to control some local network only home automation devices and currently I have pocketsphinx running on a beaglebone black but have some tuning issues to overcome...

    Of course amazon et all will pish all over their good priniciples and aims, but there's still a small chance to carve out out a niche for privacy respecting alternatives because of these guys/gals work.

  17. JohnFen Silver badge

    Re: Yeah

    "For that they should be at least given a small amount of the benefit of the doubt that they were being sincere."

    I don't doubt their sincerity, I just don't have as much faith that corporations will be able to resist doing it the wrong way as they do.

  18. Sir Runcible Spoon Silver badge

    Re: Yeah

    Of course amazon et all will pish all over their good priniciples and aims, but there's still a small chance to carve out out a niche for privacy respecting alternatives because of these guys/gals work.

    Fair point, well made.

  19. vir Silver badge

    Nope

    "There is no way to recover the audio"

    ...until another team of researchers discovers a way to contextually reconstruct spoken phonemes with 99% accuracy.

    Great "use cases" guys. Alerting you when your washing machine is done? There's already a 100dB buzzer on it for a reason. And, correct me if I'm wrong, but they already make wi-fi enabled washing machines if you want it to post to your Twitter feed.

  20. Sureo

    How is this a good thing? Who wants it in their home? The examples cited are certainly not worth the bother or expense. Miscreants are more likely to turn your smart faucet on, if you are foolish enough to have one.

  21. Anonymous Coward
    Anonymous Coward

    I'm sure these crazy inventors would be completely mystified by the feeling of revulsion that I get at the very thought of having such a device in my home.They are living in a completely different world.

  22. ma1010 Silver badge
    Mushroom

    I'm sure these crazy inventors would be completely mystified by the feeling of revulsion that I get at the very thought of having such a device in my home.They are living in a completely different world.

    THIS^ We keep hearing about more and more ways that these "boffins" are trying to come up with better ways to spy on people. Why don't all you twats of this sort out there work on something useful or, failing that, just fuck off and die?

    Any device in my home that turns out to have a microphone on when I'm not specifically talking to it will get fixed with a hammer.

  23. ThatOne Bronze badge
    Big Brother

    More accurate spying - Who wouldn't want that?

    It's a good thing for them, because they will be able to sell it for good money to Google/Amazon who will employ it to improve their profiles.

    (And obviously also for those who will use it to collect "leverage" on certain people.)

    But, as the article notes, we're just a small bunch of old fogeys always complaining; The younger generation is quite adamant about their right to be spied upon. Oh well.

  24. jgarbo

    Sorry, they are living in the real world. It's here and growing. You are the one left behind.

  25. Teiwaz Silver badge

    Sorry, they are living in the real world. It's here and growing. You are the one left behind.

    I just saw a youtube ad for Smart cameras for your home, billed as a must have for security of your home....

    Lots of enticing pics of well-furnished empty rooms. It's a damn potential showroom and shopping catalogue for would be home invaders.

    Tap that, and you can case a nice house from the safety of your own, and know when and how long the residents will be out and how vulnerable they are....

    We've long gone beyond real world, we're in a fantasy at this point

  26. Pen-y-gors Silver badge

    Boffins?

    @ma1010

    These people give mad boffins a bad name. Why can't they just concentrate on the good old hold-the-world-to-ransom-with-nuclear-weapons-in-a-volcano approach and leave our privacy out of it?

  27. Voyna i Mor Silver badge

    Re: Boffins?

    Currently, as a retired boffin, I feel that if we were in a position to hold the world to ransom with nuclear weapons in a volcano, it would be an improvement. We'd simply demand that all the politicians stop fighting and promoting stupid policies, spend the defence budgets on space exploration, fixing climate change and food security, and get on with doing something more interesting.

  28. desht

    "Sorry, they are living in the real world. It's here and growing. You are the one left behind"

    - said the turkey as he cheerfully marked an X beside the box marked "Christmas".

  29. bombastic bob Silver badge
    Big Brother

    Re: More accurate spying - Who wouldn't want that?

    "The younger generation is quite adamant about their right to be spied upon"

    they have grown up on reality TV... and post their mundane activities on social media like anybody else cares about their bowels or whatever.

    yeah, and if one o' those hears ME sneeze it'll sound like... "Ah, Ah, AH SHIT!!!" [yes I really do that, I hate sneezing - it interrupts what I'm doing]. I wonder if it'd call an ambulance, thinking it means "I fell and can't get up" (and of course THAT introduces the topic of false alarms generated by a big-nanny spy-on-your-life system). And no, I don't need Alexa to order me some narcoleptic cold medicine either.

  30. Michael Wojcik Silver badge

    We keep hearing about more and more ways that these "boffins" are trying to come up with better ways to spy on people.

    That research will happen regardless. It's much better when it's made public, as in this case, than when some giant corporation keeps it to itself. This way we at least know what's been done in the field, and can consider how to use or counter it.

  31. Steve K Silver badge

    Wire Tap

    That would be a real wire tap...

  32. Jay Lenovo
    Trollface

    <Insert porn take here>

    "Harrison said a related project called Vibrosight, which involves using a laser to measure physical vibrations of an object to determine what it's doing, has already achieved sufficient accuracy for deployment."

    Vibrosight? ..really.. Does it need a couple of AA batteries?

  33. J. R. Hartley Silver badge

    The title is no longer required.

    Ah, the unmistakable sound of anal.

    That was going to be the name of one of my books.

  34. Sir Runcible Spoon Silver badge
    Coat

    Re: The title is no longer required.

    Not 'Salmon Fishing in Fly's' then?

  35. bombastic bob Silver badge
    Coat

    Re: The title is no longer required.

    putting a new 'spin' on audio 'analysis'

    OK - that was just *BAD*

  36. SVV Silver badge

    Time to get the collection of old BBC sound effects LPs out again

    A couple of hours work in the audio editor and quick and easy distribution on YouTube can convince Jeff's little snoopbot that most people live in bizarre alien infested factories with opera singers, jet aircraft, laser guns and church bells, where doors slamming and creaking reveal populations of sinister schoolchildren, marching bands and herds of geese.

  37. Randy Hudson

    “activities and and context”

    The grammar checker I used 30 years ago would check errors like this, and that was for stuff only my teacher would be reading. Is this a tech site?

  38. Rustbucket

    Recent Events.

    So if it overheard, purely hypothetically, the sound of someone being tortured and murdered in an embassy, it could identify that?

  39. jgarbo

    Re: Recent Events.

    Yes. But it has diplomatic immunity.

  40. Rashomon

    Re: Recent Events.

    What exactly does a Saudi bone saw sound like?

  41. Marketing Hack Silver badge
    Big Brother

    Alexa, you perv....

    "On Tuesday, August 24th, I logged Mr. and Mrs. Marketing Hack having marital relations. Based on the excited chipmunk sounds Mrs. Hack made during this encounter, I sent a recommendation to our ad servers that in the future she be served with the commensurate battery of online pet store advertising."

  42. Anonymous Coward
    Anonymous Coward

    Re: Alexa, you perv....Targeted ad servers

    "On Tuesday, August 24th, I logged Mr. and Mrs. Marketing Hack having marital relations. Based on the excited chipmunk sounds Mrs. Hack made during this encounter, I sent a recommendation to our ad servers that in the future she be served with the commensurate battery of online pet store advertising."

    Well, from what I've seen of these annoying "targeted ads" that just show you things you've already purchased or own already, I predict that Alexa will just show Mrs. Marketing Hack more of the same old same old, which in this case would be ads for Mr. Marketing Hack.

  43. onefang

    "What CMU's comp sci types have added is a sophisticated sound-labeling model trained on high-quality sound effects libraries, the sort used in Hollywood entertainment and electronic games."

    I wonder if they realise that those sorts of sound libraries are often built with fake sounds? Look up what it is that foley artists actually do to to create sound effects. Some examples from Wikipedia -

    * Corn starch in a leather pouch makes the sound of snow crunching.

    * A heavy staple gun combined with other small metal sounds make good gun noises.

    * Burning plastic garbage bags cut into strips makes a realistic sounding candle or soft non-crackling fire when the bag melts and drips to the ground.

    That last one is fun. Is the device hearing you burning some candles for a romantic interlude, or is your kitchen garbage bin on fire? Having the fire brigade turning up at your tenth anniversary dinner might not be a good idea.

    * Frozen romaine lettuce makes bone or head injury noises.

    * Canned dog food can be used for alien pod embryo expulsions and monster vocalizations.

    Soooo, is your home under alien attack, or are you feeding your dog?

  44. Uffish

    Foley Fake Sounds

    The Hollywood Foley artists are to be congratulated on the accuracy of their generic sounds.

  45. Michael Wojcik Silver badge

    I wonder if they realise that those sorts of sound libraries are often built with fake sounds?

    Irrelevant. They build the model, then they test it against the gold standard, which in a case like this is typically human judges.1

    If the system does well relative to the standard, then it doesn't matter whether it was trained on authentic or synthetic data. The proof of the pudding is in the eating.

    1Hopefully a representative pool of them, tested using a methodologically-sound approach, so the baseline is useful. I'm just assuming that here.

  46. onefang

    "Irrelevant. They build the model, then they test it against the gold standard, which in a case like this is typically human judges."

    The problem there is that the human judges are being trained on the same fake sounds that the entertainment industry supplies for the models.

  47. Neil Barnes Silver badge
    Facepalm

    "...and potentially nudge users towards healthy behaviors..."

    Because I always do what teddy says...

    Harry Harrison thought of it first, if I'm not mistaken.

  48. bombastic bob Silver badge
    Devil

    Re: "...and potentially nudge users towards healthy behaviors..."

    "Harry Harrison thought of it first, if I'm not mistaken."

    He thought up 'cheddite' (Star Smashers of the Galaxy Rangers)

  49. Anonymous Coward
    Anonymous Coward

    I really wonder what goes on inside these companies

    I'm trying to find words to express how completely unacceptable I find these new features, and failing miserably. Who the f**k decides that these kind of features are 'acceptable' ? Who decides that listening and trying to make sense of people's private conversations to sling another ad (yeah right) is an acceptable activity for a business.

    And what's even more bewildering is that the American public, with their endless rhetoric about their right to bear arms to protect their constitutional freedom accept this behavior from their companies.

    Someone should maybe explain to them that the amendment providing them this freedom was not thought up so kids could walk into their schools and start murdering their fellow pupils because they wear the wrong trousers, but to protect them from bent governments and people trying to take away their right to self determination.

    I would expect a small army of minigun wielding freedom fighters would've taken up position in front of Amazon, Google et al HQ's by now.

    I'm pretty much convinced that the likes of Samsung, Huawei, Nokia and every other phone maker have this capability built into their handsets but are just not telling us.

    Pete

  50. Francis Boyle Silver badge

    Re: I really wonder what goes on inside these companies

    If (FeatureIsProfitable == true)

    FeatureIsAcceptable = true;

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2018