US schools' IT systems drop out after weekend firmware update misery A botched network switch firmware update caused school IT systems to fall over in the US state of Washington, it is claimed. WSIPC, which provides technology and software for classrooms in the Pacific Northwest, said its Skyward and IEP Online services were both out of commission for three and a half days this week while it …
Dell Co-Pilot is the support group in charge of the Dell SC Series Storage line. I would have to guess that the Dell Networking equipment was their iSCSI Switches and it dropped everything. I have a lot of experience with the SC Series and a lot of the time it is because the fault domains were not configured correctly. I see this a lot with customers that were EQL customers and switch to the SC series when the EQL was phased out. The two systems configure fault domains completely differently.
Dell Co-Pilot is the support group in charge of the Dell SC Series Storage line. I would have to guess that the Dell Networking equipment was their iSCSI Switches and it dropped everything. I have a lot of experience with the SC Series and a lot of the time it is because the fault domains were not configured correctly. I see this a lot with customers that were EQL customers and switch to the SC series when the EQL was phased out. The two systems configure fault domains completely differently.
I am guessing that the systems rebooted with a shagged/incomplete image and did not boot because the image was no good and remained unbootable. Possibly similar to Cisco ROMMON but with no TFTP or x-modem (gasp!) recovery methods.
Almost like updating your phone when it warns you not to reboot it or allow it to lose power during an update due to fear of bricking.
Either way, you're right, it's an ordinary state of affairs, but its ok because Dell was cheaper than the previous vendor.
Probably by having all memory devices writeable/upgradeable and no minimal dumb bunny firmware chip and serial IO device to allow you to start from scratch.
Bearing in mind that a minimal boot ROM and a serial port make the device potentially insecure. If you can bypass all the higher level firmware you can both recover from and cause a major incident.
I heard that some of the recent updates to a SC Series enforce TCP/IP setting that can cause problems with iSCSI even if it was working before. Older switch setting would work fine with the older SC Series version, but as soon as the first controller is upgraded and active again it would probably fail immediately. I figure it would probably be hard to diagnose if you had not seen this before. Once they figure it out they would need to make the proper changes to their existing switch infrastructure or they would have to upgrade or replace it.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
