Obligatory HAL reference
Beyond that, from sound of it training mainly with "face on" speakers in news etc, would be interesting to see how well it copes with less ideal viewing angles.
LipNet, the lipreading network developed by researchers at the University of Oxford and DeepMind, can now lipread from TV shows better than professional lipreaders. The first LipNet paper, which is currently under review for International Conference on Learning Representations – ICLR 2017, a machine learning conference, was …
No. How it copes with non-native speakers.
This is something which was used by the Nazis in WW2. A deaf lip reader will immediately notice if someone is non-native speaker even if his language and pronunciation is so fluent that nobody notices while listening to him.
There is more than one way to form sounds - especially accented vowels and the various hissing sounds in anglo-saxon languages. Due to the differences in muscles, etc and even things like milk teeth vs grown up teeth learning them as a child results in different mimics compared to learning them as an adult.
"This is something which was used by the Nazis in WW2. A deaf lip reader will immediately notice if someone is non-native speaker even if his language and pronunciation is so fluent that nobody notices while listening to him."
Given that the Nazis sterilised deaf adults merely for being deaf and murdered deaf children merely for being deaf (with the cooperation of some deaf schools), this sounds remarkable; do you have a source?
(A brief google turned up this: http://forum.axishistory.com/viewtopic.php?t=91238 )
Given that the Nazis sterilised deaf adults
Most likely same as the Soviets. There were different rules inside and outside the house. KGB ran full scale research on radiology and genetics while officially supporting the party line that this is an invention of the Capitalist propaganda.
The Gestapo employed a small number of deaf people who had professional lip reading ability exactly because the best lip readers are deaf. I have come across references to this in both Soviet and Western historical works so I have no reasons to believe it is not true (Soviets were lamenting that they had a couple of agents in Poland picked this way). It's been a while - I came across this 20+ years ago so cannot remember the exact source.
Apparently this is how it's done...