Drive Firmware
"The Reg imagines readers will be keen to know which company's kit gets corrupted firmware when SANs crash"
They're not necessarily talking about corrupted firmware. I don't know any specific details about the FUBAR implementation at the ATO, but many years ago when I still did hardware I worked on DEC kit containing SWXCR RAID controllers, which were part of the DEC Storageworks tech taken over by Compaq and then HP.
I remember once having to flash a customer's drives' firmware due to firmware issues (probably DEC branded Seagate wide SCSI back then), because if there was a mismatch between what a drive was doing and what the controller thought it was doing, under certain circumstances the controller could mark a perfectly serviceable disk as bad and drop it from the array, and however much swearing and jumping up and down you did, it would refuse to mark it good again and bring it online and back into the array (unless you reinitialised the disk, wiping the data.)
In a failure scenario, where you've got other real hardware errors, this is disastrous as you can lose the whole array (and kiss your data goodbye.)
At this point you find you need a clean pair of trousers, and discover just how good your customer's DR strategy is...
Biting the hand that feeds IT
