back to article XtremIO 'outages bork US hospital patient records system'

XtremIO storage is reportedly failing at Fairview Health Services in Minnesota, hindering patient operations. Fairview is a non-profit organisation with around 25,000 employees in hospitals, clinics, care homes and pharmacy shops. It traditionally used HDS storage for medical records used by its Epic patient admin system, and …

  1. Anonymous Coward
    Anonymous Coward

    They are not alone

    Our backoffice provider, a Fortune 500 firm, has had two XtremIO outages - both firmware-related - that have taken them offline for nearly 20 hours this year. I'd be scared shitless if I had mission-critical systems relying on that storage.

  2. Anonymous Coward
    Anonymous Coward

    From http://www.citypages.com/news/whistleblower-fairview-health-services-it-system-keeps-crashing/399552011

    ...a veteran Fairview IT worker, says under its old storage system with Hitachi, the hospital chain had one across-the-board IT system outage in 12 years. Since switching to EMC in fall of 2015, it's had three crashes in one year.

    1. Anonymous Coward
      Anonymous Coward

      blanket statements like that...

      ...are usually dubious

  3. Adam 52 Silver badge

    "Epic failures compromise safety"

    Why, why, do people keep repeating this. Have you ever been to a hospital? Anything important is printed out, written down or riveted to the patient. The computers just make the admin easier but the procedures are designed so that losing them doesn't make anything unsafe.

    1. Gene Cash Silver badge

      Yes, I've been to a hospital... where they confused my broken collarbone with a broken ankle and tried to make me use crutches.

      "These x-rays say your ankle is broken"

      "No, those x-rays say Jeff -------'s ankle is broken."

  4. Anonymous Coward
    Anonymous Coward

    What did they smoke before switching from HDS to XtremIO ? Who does that. HDS storage is gold standard for mission-critical apps.

    1. Anonymous Coward
      Anonymous Coward

      As many on here have probably suspected from time to time in other cases perhaps a VP or board member has buddies or family at EMC. Plenty of studies say the ethically challenged do tend to be over represented at the top.

      1. Anonymous Coward
        Anonymous Coward

        Not to mention,

        In the case of my Fortune 500 friends above, they were already an EMC shop so I suspect the EMC sales guys had some pretty strong motivations for keeping it that way. Better to sell them something crap with an EMC label on it then let someone else get a toe in the door.

    2. kotaKat

      Unless Epic pulled the "you will upgrade your end of life'd storage to XtremeIO" line.

      I was told by a coworker, and I don't know how true this was, but that Epic shops weren't run by the local shop on implementation; Epic told you what to buy and who to hire and what to do; if you didn't play along you lost points on a "quality of installation" score which directly impacted how much you paid for maintenance and support.

  5. Anonymous Coward
    Anonymous Coward

    XtremIO is the Samsung Note 7 of the storage industry

    Seriously, with all of the anecdotal stories out there of major XtremIO outages, you are either on the take or just not that smart if you buy into it. XtremIO is the Samsung Note 7 of the storage industry.

    1. Anonymous Coward
      Anonymous Coward

      Re: XtremIO is the Samsung Note 7 of the storage industry

      They wouldn't let me on the plane with it either.

  6. Anonymous Coward
    Anonymous Coward

    APAC too

    i know of 3 xtremeIO outages which resulted in either total or major data loss. primarily due to firmware update issues.

    1. Anonymous Coward
      Anonymous Coward

      Re: APAC too

      Not just firmware .. unexpected power failure also seemed to cause XtremeIO to lose the plot. If you lose the in-memory metadata, on a pair of nodes the whole cluster bricks. I've been told of this happening in multiple unrelated installations, with subsequent offers of bargain basement all flash VMAX's (which are probably a better fit for EPIC in any case) coming from embarrassed EMC account teams desperate to retain the business

  7. Sir Sham Cad

    No outages so far

    But our VDI estate lives on XtremeIO so now I'm fucking bricking it that my Xbricks will, er, brick it.

  8. NRW61

    The City Pages article was inaccurate and misleading across a wide range of facts, including the comments relating to the Dell EMC products. The Dell EMC products never went down or failed at any point. XtremIO was not the root cause of any incidents referenced in the article.

    1. Anonymous Coward
      Anonymous Coward

      So Fairview is lying?

      EMC can make statements, but clearly Fairview's own internal documentation says otherwise. From http://www.citypages.com/news/whistleblower-fairview-health-services-it-system-keeps-crashing/399552011:

      According to internal Fairview documents, glitches related to the EMC storage system are limiting care givers' access to Epic, a data system in use at Fairview and many other American hospitals. Epic's applications are responsible for everything from registering a patient and scheduling blood work to fulfilling pharmacy orders.

      --

      On September 1, "a major outage was declared" just after 9 a.m., an email written later that same day by (Don) Tierney acknowledges.

      "I'd like to begin by recognizing and apologizing for the difficulties this — and all — system outages cause," it says. "We know outages cause tremendous complications related to patient care and satisfaction, and for many of you, they make your jobs more difficult.

      "Today's event was a result of too much activity occurring on recently implemented storage system."

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019