"so any savings have to cover that"
They generally don't. If you've got the scale to operate any process yourself, then why will an outsourcer be cheaper? As you say, they will have crappy conditions, pay and pension, but that's offset entirely by the costs of bidding and set up, risk pricing, infrastructure, net margin, and overheads (outsourcers usually have vast amounts of goodwill on their balance sheet to cover). And there's no real operational time savings - outsourcers don't offer you a new, magically more efficient process, they want to take your under-performing process and implement that with lower paid staff (hopefully with lots of "unforeseen" problems to be fixed at great cost).
In general, the outsource provider needs to charge 110 to 120% of your pre-outsource cost to cover their costs and make their target margin. But to win the work they need to be cheaper, with a rule of thumb that says "bid 20% below the existing cost base". So for a five year contract, in year one the outsource provider delivers the 20% saving, accrue a loss and add a further hangover to be recovered. By the end of year 2 they need to start getting into the black, and hopefully the client has some changes that will enable them to be fleeced. If not, then rely on the fact that the outsourcer's commercial people write contracts day in, day out, and will run rings round the buyer. Either way, the outsourcer assumes that the client will walk at year 5, and so simply reams the client out, costing on average 40% more than the inhouse cost in years 3 through 5. There was some good research on this published some years back, but it was quietly hidden when the analysts who published it realised there was far more loot to be garnered by trumpeting the "benefits" of outsourcing, rather than telling people it was doomed to fail. At year five, it the vendor is really lucky, the buyer will have forgotten the original business case, and just renew. Otherwise, chances are they will be too fearful and shamefaced to bring back in house, in which case the work stays outsourced, and the outsource sector feed off each others churn: "Rinse and repeat".
Directors are paid enough, and should know these things. Unfortunately they listen to idiot management consultants, or believe the vendor's sales pitches, ignore common sense, and do it anyway. My own employers are at the point where our IT outsource "partner" have given up trying to improve their dismal service, and are now moving firmly into "monetise the client" mode. All forseeable, unless you're paid several million euro a year, and sit in a boardroom in some ivory tower.
Obviously different rules apply where you don't have the scale or competence to do something yourself, and in those cases outsourcing can be a sensible move - although usually something better suited to SME's rather than big corporates.