"It is an intelligent cruise control system, not a self-driving car. The driver should always be concentrating and be ready to take control back. If they aren't, they are driving without due care and attention.:"
Well, yes. But the problem with that -- and this has also been shown to be a problem with pilots -- is that when you take over the human's job for a while, when they no longer have to provide inputs, their attention wanders. Even if they don't start physically doing something else, their mind is no longer on the task of hand, because it's being done for them. It takes tens of seconds to come back from that.
In airplanes they attempt to deal with this with training, and it also helps that things usually unfold pretty slowly at 30,000 feet. There's very little to hit up there, and very few situations require immediate action. Still, there are several accidents where the autopilot decoupled due to a problem with the airplane, and pilots acted inappropriately due to not being in the loop about what was happening.
Biting the hand that feeds IT
