One-dimensional, hence exceedingly fragile
Without delving into technical details, it looks to me as though the designers of those systems have made the mistake of treating the challenge as no more complicated than the Turing Test.
As has often been pointed out, to pass the Turing Test a computer does not have to "think", "feel", or in any way simulate the operation of a human nervous system. All it has to do is, on one single occasion for a limited time, manage to hold up its end of a conversation in such a way that its interlocutor cannot distinguish it from a human being.
The pattern recognition systems designed so far apparently aim only to meet certain performance criteria under "normal" conditions. They do not seem to have been designed to cope with unusual or adverse conditions. They need to be subjected to the ministrations of a Tiger Team - people who will go to great lengths to make them fail. Only if they can be shown to go on working reliably regardless of such adverse conditions can they be considered as even eligible for safety-critical tasks.
Biting the hand that feeds IT
