XtremIO 'outages bork US hospital patient records system' XtremIO storage is reportedly failing at Fairview Health Services in Minnesota, hindering patient operations. Fairview is a non-profit organisation with around 25,000 employees in hospitals, clinics, care homes and pharmacy shops. It traditionally used HDS storage for medical records used by its Epic patient admin system, and …
From http://www.citypages.com/news/whistleblower-fairview-health-services-it-system-keeps-crashing/399552011
...a veteran Fairview IT worker, says under its old storage system with Hitachi, the hospital chain had one across-the-board IT system outage in 12 years. Since switching to EMC in fall of 2015, it's had three crashes in one year.
"Epic failures compromise safety"
Why, why, do people keep repeating this. Have you ever been to a hospital? Anything important is printed out, written down or riveted to the patient. The computers just make the admin easier but the procedures are designed so that losing them doesn't make anything unsafe.
Unless Epic pulled the "you will upgrade your end of life'd storage to XtremeIO" line.
I was told by a coworker, and I don't know how true this was, but that Epic shops weren't run by the local shop on implementation; Epic told you what to buy and who to hire and what to do; if you didn't play along you lost points on a "quality of installation" score which directly impacted how much you paid for maintenance and support.
Not just firmware .. unexpected power failure also seemed to cause XtremeIO to lose the plot. If you lose the in-memory metadata, on a pair of nodes the whole cluster bricks. I've been told of this happening in multiple unrelated installations, with subsequent offers of bargain basement all flash VMAX's (which are probably a better fit for EPIC in any case) coming from embarrassed EMC account teams desperate to retain the business
EMC can make statements, but clearly Fairview's own internal documentation says otherwise. From http://www.citypages.com/news/whistleblower-fairview-health-services-it-system-keeps-crashing/399552011:
According to internal Fairview documents, glitches related to the EMC storage system are limiting care givers' access to Epic, a data system in use at Fairview and many other American hospitals. Epic's applications are responsible for everything from registering a patient and scheduling blood work to fulfilling pharmacy orders.
--
On September 1, "a major outage was declared" just after 9 a.m., an email written later that same day by (Don) Tierney acknowledges.
"I'd like to begin by recognizing and apologizing for the difficulties this — and all — system outages cause," it says. "We know outages cause tremendous complications related to patient care and satisfaction, and for many of you, they make your jobs more difficult.
"Today's event was a result of too much activity occurring on recently implemented storage system."
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
