back to article Don't shout at your disk drives, warns Sun engineer

Those among you who are prone to vocal outbursts of rage at cantankerous kit should be aware that hardware has feelings too, and directing your ire at disk drives can provoke "unusually high disk I/O latency". That's according to Sun's Fishworks engineer Brendan Gregg, who explains just how to upset your drive on his blog and …

COMMENTS

This topic is closed for new posts.
  1. Simon Ward
    Coat

    Don't get mad ....

    ... get a hammer.

    That is all.

    Mine's the one with the 7lb lump hammer marked 'Fine Adjuster' in the pocket

  2. Anonymous Coward
    Thumb Up

    Quite right too...

    ...shouting and verbal abuse is reserved for users, percussive maintenance is for hardware.

  3. mittfh
    Coat

    Hammers...

    I always find 14 lb ones, available from most DIY stores, are more effective (and satisfying)...

    Just don't forget to buy a pair of safety goggles while you're at it...

    Alternatively, "earth" your equipment to the building's lightning conductor and wait for a storm, cut a hole in the roof before ordering a skylight, etc....

    Mine's the one with a copy of that entertaining read "50 ways to shut down or crash a Netware Server" in the pocket...

  4. mittfh
    Flame

    Talking of which...

    http://packetstormsecurity.org/Netware/penetration/novell-crash.html

    Some of the suggestions in November's "Back the F:\ Up" thread would also be satisfying (e.g. initiating a thermit reaction on the equipment - hence the icon).

    And if your server uses RAID, you can hot swap a healthy disk in to solve the immediate problem, then have fun "fixing" the dodgy disk...

  5. Anonymous Coward
    Thumb Up

    Real enginnering

    At last, some common sense engineering experimentation for 2009 instead of some whacky partial financial modelling from those crazy maths guys...

  6. Wize
    Boffin

    I find a bit of percusive engineering works wonders

    Nothing teaches new hardware to work properly than sitting the unit it replaced beside it, after you've stuffed it with oily rags and burnt the misbehaving unit out as an example to all.

    Muhahahahahahaha

  7. Simon Williams
    Unhappy

    It's pickin' up bad vibrations

    Is anybody concerned that this guy works every day in an environment this noisy without any form of hearing protection. He could probably lower the latency of the drives still further IF THERE WASN'T SO MUCH BLOODY NOISE IN THE LAB IN THE FIRST PLACE.

  8. EdwardP
    Flame

    Toture...

    When a machine really deserves it, I personally like to switch the power supply (on back) to the wrong voltage.

    With the right PSU your box will make a screaming sound and begin to spew white smoke from the back. Now that's satisfying.

  9. David Shepherd

    @get a hammer

    that used to be (semi-)official support advice from Sun engineers ... the original Sun4 SparcStation had hard-drives that fairly regularly would get stuck in the "head park" position if they were powered down. The advice to fix this was to give the unit a sharp hit on the side to dislodge the heads!

  10. Dunstan Vavasour
    Thumb Up

    Instrumentation

    Of course, while the effect of shouting at your discs and causing vibration is quite diverting, the point of this video is to demonstrate the detailed instrumentation available in these new 7x10 arrays.

    The UI not only shows the momentary increase in latency, it identifies which spindles this is occuring for. When you remember how inexpensive these arrays are, this is an extraordinary level of detail. Oh, and this is available out of the box, it's not an optional extra.

  11. Simon Ward

    Re:Quite right too...

    "...shouting and verbal abuse is reserved for users, percussive maintenance is for hardware."

    If only it could be the other way around. If only ...

  12. Anonymous Coward
    Paris Hilton

    Device above

    I wonder what the device is above the array in question ?

  13. Peter Hawkins

    Re Device above

    It's the head unit for the 7410. It's actually a 2U AMD based server.

  14. Bill
    Happy

    To Simon and AC

    "It's pickin' up bad vibrations"

    - Come now Simon... Who works all day in the lab? That's what remote access is for. Of

    course it's possible that this guy does, but I doubt it. Your other comment on reducing the

    level of noise in a lab... Have you ever been in a "real" data center? They are loud! The air

    conditioning alone is loud, but then when you add a ton of spinning disks and server fans to

    the mix it's incredibly loud. The lab shown in this video is not any louder than the average

    DC (try putting a mic in your data center and see how loud it sounds). Ear protection is a good

    idea though. I know many old mainframe guys that are now deaf because of data center

    noise. I often spend countless hours in data centers without hearing protection and I doubt

    that I am alone in that regard.

    "I wonder what the device is above the array in question?"

    - That's the actual 7440 NAS Head. Then above that is another JBOD hooked up to another

    7440 NAS Head. You can check them out on Sun's website.

    I for one am very impressed with the level of detail shown by the analytics of this box. Very impressive! EMC, HDS, and NetAPP don't even provide this level of detail in such an easy format for fee.

  15. Anonymous Coward
    Paris Hilton

    Re: Device above

    It's not the head unit for the the 7410.

    The 2RU unit for the 7410 is a x4440. The device in the picture is 1RU, and has 4 non drive modules

    in the front.

  16. Anonymous Coward
    Paris Hilton

    @To Simon and AC

    "Have you ever been in a "real" data center? They are loud!"

    Not as loud as the silence when the power and the UPS fails. That is deafening.

    Paris, 'cos she likes a good back up.

  17. John Angelico
    Coat

    It's a fundamental design problem...

    The basic mathematics of binary computing is the cause of it all.

    Subtraction is done by two's-complement addition (it's faster).

    But two's complementation requires a bit flip to be propogated through an entire byte value. This often results in a carry at the top end, which either goes to the low order if that's a zero, or has to be dropped.

    These spare bits left over simply crash to the floor, which is why there is so much noise in server rooms, and the poor hard drives can only take so much vibration.

    Consequently, they are hyper sensitive to any additional vibration induced by our angry vibes.

    Yes, mine's the one with the bunch of April dates hanging out of the pocket.

  18. Anonymous Coward
    Anonymous Coward

    @David Shepherd

    That approach might have fixed one of my drives which had stuck heads, if I had known the problem before opening it. The difference being that the heads were stuck to the platters. I don't think the motor had enough torque to free them – it took quite a bit of force by hand. Nothing particularly important on that drive, but I make better backups now.

  19. Anonymous Coward
    Anonymous Coward

    re: @To Simon and AC

    "not as loud as the silence when the power and the UPS fails"

    You've never heard a UPS alarm in close proximity then? You're right, you won't hear much after that. Except a high pitch whistling noise

  20. John Browne
    Thumb Up

    That might explain it.

    A couple of years ago, contractors were installing attachments for a false ceiling on the floor below mine, using hammer drills. I could feel the vibration through my feet, my chair and my desk.

    I got the distinct feeling my PC was running slowly, but thought at the time that it was just the racket making me impatient and affecting my thinking. It was driving me up the wall.

    My system box was on the (vibrating) floor, so this research could provide the real answer, and I was not going crazy.

    Phew! Thanks for that.

  21. vincent himpe

    To paraphrase Beldar Conehead

    Sure, and let me know when elvis gets here ...

  22. Anonymous Coward
    Coat

    @John Angelico

    Close, but wrong. The problem does lie with the binary nature of computing, though.

    The issue isn't so much with the zeros as it is with the ones. Zeros, being round and smooth, flow along just fine. The ones, however, knock about, tumble, and get jammed together, resulting in noisy data flow (just like water in a pipe). Today's faster computers with their smaller conductors in all the ICs compound the problem.

    The old curly keyboard wires were really notorious for getting the ones stuck in them, which is why you tend to see the non-curly wires on modern equipment.

    Mine's the one with the BOFH excuse of the day calendar in the pocket.

  23. Turgut Kalfaoglu

    loudness

    This probably also means "don't put speakers next to your PC" and "Don't turn the volume up too high"?

  24. Secretgeek
    Joke

    If it doesn't work...

    ...hit it with a bigger hammer. If it breaks, it needed fixing anyway.

    Motto of engineers everywhere.

  25. Yorkshirepudding
    Go

    maybe

    the magic smoke escape while you had your back turned while going for the hammer

    stick that in your drive bay and format it!

  26. Bob Miller
    Alien

    re: two's-complement addition

    John,

    Is two's-complement addition really faster, or is does it just require fewer transistors (reuse those for addition when doing subtraction)? At one time minimizing transistors was very important, but today the question is usually how can you best use the available ones.

    Is it time to reevaluate the choice between one's complement and two's complement arithmetic? A one's complementation (nice verb) is a simple XOR operation and does not require a bit flip to be propogated through an entire byte value. This would mean no spare bits left over to crash to the floor. Perhaps the noise associated with these bits was not as significant years ago when the processors were several orders of magnitude slower.

  27. Steve Liddle
    Happy

    foot work ?

    I find kicking the case often makes the clicking noises go away, usually a lose cable catching on a fan, soon goes away, the last drives that failed got some large nails pushed thru with a sledgehammer as not want personal data to re-appear :)

  28. Scott F. Gunelius
    Paris Hilton

    @Simon Ward

    If only I could forcechoke some of the users or do a "Scanners" number on them, I wouldn't need percussive maintenance. No physical evidence either...

    Paris, because forcechoke wouldn't work on her, her throat muscles are too well disciplined

  29. Anonymous Coward
    Anonymous Coward

    Hard Drive

    Am I the only one who remembers having to gently warm IBM drive spindles with a lighter to get them to spin up after a long shutdown?

  30. quartzie
    Boffin

    Re: Hard drive

    Probably, everyone else died when they got doused by an overzealous fire control system.

  31. Seán

    re: two's compliment

    I had to laugh when I read the posts on two's compliment ooperations as both of the previous posters ignored the endedness problem as though it didn't even exist! In my day we had to balance the big endian rigs with the little endian ones so's they complimented each other. We were young in those days and no problem was insurmountable, when I see the kids these days practically counting on their fingers I despair.

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2019