And then there's the staggering lack of competence
As others have pointed out, a dedicated security team rapidly becomes a major obstacle to getting anything done. It's much easier to say 'no' than to make your job hostage to some failure, however minor and inconsequential, that can later be used against you.
And that problem is multiplied ten-fold when your security team is poorly managed and staffed by people who don't actually understand the subject very well. If IT in the Anglophone world has a chronic problem with cowboys, then—barring web development, which seems to be infested with incompetents—security is like the seventh circle of hell. There is something about it that attracts a certain kind of personality: very often those political types who love the sound of their own voices, enjoy a few scraps of power, and still labour under the delusion that some hideous Powerpoint slides with naff toons nicked from the web is in some way equivalent to providing management, leadership or "doing strategy".
There is no such thing as perfection, and security is never perfect. It's always about trade-offs. This requires calculation and judgement. Don't spend £1m bulletproofing your business against a mythically unlikely attack that would only cost you a few grand even if it occurred. Conversely, don't let bean-counters deny you the £100k you need to ensure that a million customers' travel habits don't get leaked on the net—just because the leak wouldn't cost the business a penny in fines or refunds doesn't mean the reputational damage won't kill you.
Barring network specialisations—right down to hardware level, because the plumbing is a special case—you should indeed discard the very concept of a security team. Instead, get your management, for once, to do something useful, in understanding the real threats and risks, distinguishing catastrophic scenarios from mere inconveniences, setting priorities, and then making sure that the folks working on the vulnerable-with-consequences systems know that their careers depend upon building security into their work, not as an afterthought but as part of its DNA. (And don't forget to train and resource and appreciate them properly, or it will all be for naught.)
As ever, it all comes down to good management and leadership by people with brains and long-term vision. Unfortunately, the current cadre of executive management is mostly short-term, greedy, makes a virtue of ignorance of detail and constitutes, in short, a Boris Johnson approach to everything ... so despite what I said, you're doomed.