Re: Testing failover.
Twelve or fifteen years ago, when the BBC rebuilt most of its radio playout systems, A studio had a local server which cached the audio from the apps room servers. The apps room had three servers using two different OSes; all these ran at once to deliver audio. In most cases there were two identical apps rooms, at opposite sides of a broadcast centre, on separate power and network systems and fed from external circuits that didn't share ducts...
I recall driving up the M1/M6 to Birmingham with *all* of Radio 2's music on a server in the back of a rented van; significantly faster than transferring it electronically (though a couple of years later we re-engineered the delivery backbones with 64Gb/s circuits).