IBM storage kit keeping admins awake at night Management software for IBM's DS3000, DS4000 and DS5000 series storage arrays is waking sysadmins at 2:00 AM, thanks to an obscure software setting. As detailed in the Aussie Storage Blog, some versions of the Profile Collector feature in Big Blue's DS Storage Manager package schedules data collections at 2:00AM. The blog's …
The problem here is software which assumes it knows best about what it installs. This ranges from putting an icon on your desktop you don't want to installing crapware or spyware which you definitely didn't want. Some software hard-codes application paths, but the worst culprit is itunes which, without your knowledge, installs a bunch of services, including one which harvests the media on your hard drive, bringing your PC to a standstill. This is the #1 reason itunes is banned on work machines.
In this case, (unlike itunes) you can deselect the offending software but it should really ask if you want to install rather than the other way round. The worst thing is the software doesn't even need an installer. A zip file would do.
Storage Manager is bloated anyway. It's mostly a JRE.
As for collecting data, you can always use a script with a scheduled task in Windows. I'd post how to do it it here but I doubt anyone really cares.
Any software which triggers a load of work at the same time on multiple systems can cause this type of problem.
And don't pick times like 1:00, 2:00, 3:00 because some other software is probably using those times (Windows update with a default of 3:00, default times used by backup products etc). Stagger them instead.
And the experienced sysadmin might collect alerts and deliver them via SMS at say 6:00 am.
If one of my admins collected alerts to be delivered at 6am, he'd be looking for a job working for someone else.
If, on the other hand, he recognized that the client application developers were (and probably still are) idiots, and reconfigured things accordingly, he'd find himself less often on-call.
"If one of my admins collected alerts to be delivered at 6am, he'd be looking for a job working for someone else."
Not so fast.
It depends on your SLAs and what cover you have with your hardware support folks. When I was an admin for a large development and testing environment we only had hardware cover from 7am onwards. A 6am alert was quite adequate to get in early and either fix things yourself or place a hardware support call.
Production systems were a different matter of course.
Being a storage admin of IBM DS4800 boxes I'm usually already awake at 2.00am worrying about the integrity and availability of the data on those buckets of dog poo. (or actually restoring the data , or installing this weeks firmware, or just crying.)
I wonder if NetApp knew what they where in for when they bought Engenio . . .
At our site, we have a renowned SAN vendor that denied this syndrome. We ran a benchmark that showed DRAMATIC dataflow reductions at the two start-times of our enterprise backup system. We saw I/O go from x to 6x in the early evening AND from x to 9x at 2AM (when the big systems did enterprise backup).
That vendor denied it, even after we showed the benchmark output. The benchmark was absolutely reproducible, too. That is what caused us to bring in OTHER vendors for SAN equipment. At least, those vendors were up-front about this kind of problem.
In my previous post, 'x' was the amount of time to do the 15GB data transfer (disk-to-disk). That same data transfer went up by a factor of '6' and '9' when the Enterprise backup system schlepped the data from local-dump files off to the Enterprise Backup system...
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
