> brake? Not quickly enough as for 1/4 second or so you are thinking "is it going to brake? Am I supposed to take over? Oh shit it's not doing to stop"
No, hell no.
You should always be taking over - braking or whatever - immediately. You should never wait to see what autopilot (or any other 'super-cruise', or pretend autonomous, or even real autonomous system) will or will not do. Those systems are there in case the driver fucks up, they are a backstop. Not the primary system. You, as the driver, are the primary control system.