two kinds of watchers to handle physical security,  human and  machine.
 security personnel are usually badly paid and badly treated; therefore, they are usually apathetic or passive-aggressive, and unwilling to exercise any initiative in their job. it is usually pretty easy to get past a human checkpoint. to address this, they need decent pay, training, effective leadership, and a stake in whatever they are guarding (so, they're more likely to win a lottery than get any or all of those).
 machines are not too good at recognition, especially biometric recognition. additionally, a machine watcher is predictable and rule-based, and incapable of initiative; therefore, ways can be found around most automated checkpoints. the watcher mechanism has to change periodically, to void possible developing workarounds; this is expensive and painful.
neither method is perfect, but a combination of the two would mostly cover the failings of either. this requires careful thought, and effective planning and implementation, with continuing maintenance and review; all of which rarely ever happens anywhere these days.
therefore, we are doomed: stoned hippy terrorists will demonstrate inside the PM's residence, with placards made from hemp, and sing "Kum ba yah".