Feeds

back to article Sysadmins fail to fix NHS IT snafu, HUNDREDS of appointments cancelled

IT staff at the UK's largest health board have failed to fix a major system error that's cutting health workers off from clinical and administrative services. NHS Greater Glasgow and Clyde (GGC) had to cancel 288 outpatient appointments yesterday along with four planned inpatient procedures, 23 day cases and 40 chemotherapy …

COMMENTS

This topic is closed for new posts.

Page:

Patients

So what are they telling their patients? Patience?

1
0

"We apologise for any inconvenience"

...or premature death

4
0
Silver badge
IT Angle

Re: Patients

How will a Gilbert and Sullivan operetta help? Or is that what they're playing while on hold? If the latter, it sounds like they really need the whole Ring Cycle?

0
0
Silver badge
Meh

"network problems"

Ah- that old chestnut.

2
0
Silver badge

Re: "network problems"

Probably a connectivity issue, as in "it's a public holiday in India today, and we can't reach any of our outsourced IT staff"

7
0
Silver badge
Mushroom

Re: "network problems"

Yes, and we can all thank Bill for that error message (no matter how many times it was used.)

1
0
Bronze badge

I told them....

once you give up quills and parchment it will all go wrong but no they went ahead and had the 19th century regardless

4
0

GG&C is a muddled mess of old health boards shoved together, they have a lot of management, team leaders etc.

I'm just curious if this is down to lack of change management with a staff member doing something silly, however as it's IT this would almost certainly be covered up with "transient network issues".

2
0

speaking from experience

everything is fully changed managed up there

0
0
Anonymous Coward

A "network" problem?

Betcha it isn't ... if by network you mean transport layer downwards.

1
0
JQW

So...

So, where are these SysAdmins based? Bangalore or Kuala Lumpur?

2
0
Anonymous Coward

Re: So...

Not sure about the sysadmins but we email Atos for our tech support.

0
0

Re: So...

also in Glasgow, I believe some of the 1st line helpdesk was outsourced at one point :)

0
0

Re: So...

Atos? Atos would say the system was perfectly healthy even if it was dead.

2
0
Holmes

Failed on 01 Oct?

Ten quid says there's a calendar somewhere with 31 days in September.

(Can we have an "only half-joking" icon please?)

7
0
Joke

Have they tried...

[chris o'dowd'] Turning it off and on again ? [/chris o'dowd']

2
0

so presumably..

the medical staff still went to work that day, and the patients still turned up, with appointment letters in hand, and were turned away because "computer says no"?

WTF, why not send them in? What have I missed?

4
0

Re: so presumably..

Exactly!

Pen + Paper anyone? Or do they use Star Trek-style EMHs all over the place?

2
0
Bronze badge

Re: so presumably..

"What have I missed?"

Maybe a large proportion of their patient records are now computerised?

1
0
Anonymous Coward

Clinical apps seem to be back up as of 5 minutes ago but email and file access is still stuffed.

0
0
Anonymous Coward

So ... AD problems then?

2
0

There really are a shed load of problems that can happen on the network.. AD just complicates it further up.

0
0

This post has been deleted by its author

Probably not, that would lock everyone out. My guess is the PAS (Patient Admin System - backbone of the whole dealio) interfaces are buggered in a not-easily-fixed way.

0
1

This post has been deleted by its author

Anonymous Coward

AD

Some Chaos Monkey deleting OUs??

0
0
Anonymous Coward

DNS

Sources close to the troubled starlet claim her DNS is in rehab.

Imagine if you will, a 20,000+ seat organisation with a DNS zone up the wazoo.

0
0
Silver badge
Meh

AD problems

So it seems.

0
0

Every time there's an issue in the cloud, there are plenty of people taking the "told you so" line. This appears to be an example of the reverse. Running your own data centre is all very nice and you can certainly touch everything, but it's very expensive indeed to provide the sort of N+1 or 1+1 redundancy that is standard fare in colo or managed data centres. I'm not taking sides here, just trying to balance the argument a bit.

1
2
Silver badge

@Dr Who

Ah, but if the problem is in-house, with your own employees, then you can at least bang some heads together and threaten to kick people out of the door.

If the problem is either in the Cloud, or with some outsourcing outfit, all you can do is threaten to not renew your contract in 4 years, by which time everybody will have forgotten!

Of course, if the contract was negotiated properly, you may get some financial recompense, but I don't trust any of out public organisations to be that good at getting such clauses into a contract....

3
0

Outages

These do happen. For me it's a bit rich to accuse of system of failing the patient because one the basis of one days downtime. I'd need to know what benefits it created the days it was running. Then I have an idea of whether this is a story or not.

If, on the other hand, they can't quantify the benefits of the system, just turn the bloody thing off of course.

2
0
Silver badge

Re: Outages @Fletchulence

Sure, outages happen. But good design should make such an outage relatively short. Keeping critical systems up even when things are failing should be part of the design and change process.

One day's down time for a regional health authority equates to hundreds of thousands of pounds of wasted resource, even more if you take into account the time of the people who attended their out-patients appointments without being seen.

I sometimes remember when I used to talk to a power engineer in a regional electricity distribution company, where five nines is a way of life, not an aspiration.

0
0
Anonymous Coward

Re: Outages

Absolutely outages do work, but it's how fast you recover from a major outage and what the cause of that outage is that's important. Users will accept, grudgingly that there will be outages, but they won't accept one which drags on with little or no information coming from IT, which is usually the case.

As someone who's been involved in investigating incidents like this within the NHS it's likely that half truths are already flying around and users are getting half of the story whilst IT management try to cover up a complete balls up. Let's hope that's not the case, for a change.

0
0
Anonymous Coward

Re: Outages

It ties all the clinical Applications together as well as the respective data, clinicians can then logon enter a users Unique ID, view information, tests etc which have been collected from various applications in different departments, etc NHSGG&C is a damn site better integrated system wise than Englands NHS

0
0
Anonymous Coward

Re: Outages

Ties them together but doesn't stop you individually logging into each system to continue doing your work and the system also shouldn't pull others down when it breaks..

0
0
Holmes

Ealing powers

Lateinos and Romiith, anyone?

The Sherlock icon... for those in the know...

0
0
Anonymous Coward

I think it's more than Glasgow.

NHSmail isn't allowing attachment downloads and Choose and Book (lovingly called Booze and Chuck) isn't working for private healthcare companies.

0
0
Bronze badge

This is what happens...

...if your sister brother's one man IT company is given the contract so that you can bank some of the costs by him giving you backhanders

0
0
Silver badge

Re: This is what happens...

If you're BillG's brother, perhaps...

0
0
Bronze badge

"they had come up with a way to let chemotherapy patients be seen"

"they had come up with a way to let chemotherapy patients be seen",

like before there was something called IT?

What exactly did the IT team do, pick up the patients in their own car and drive them to see the doctor?

0
0
Anonymous Coward

Re: "they had come up with a way to let chemotherapy patients be seen"

I think you're missing the point, boards in Scotland are increasing going towards paperless or at least paperlight working, meaning there's nothing to refer to . for example if you're going to give someone radiation treatment it's quite important to do so in the correct location not with the patient pointing and saying "around here" and with the correct dose etc.

0
0
Anonymous Coward

"So...AD Problems then?"

Without going into any further detail, I can confirm that this is where the problem lies...

3
0

Err hold on - surely the patients know when their appointments are - all the docs have to do is sit in their office and wait for em...

1
0
Anonymous Coward

...

.. Blue screen of actual death???

3
0
Silver badge
Coffee/keyboard

Re: ...

That was my keyboard.... you bastard.

1
0
Bronze badge

Same old, same old

So a poorly planned system that's had a thousand political fingers screwing up the design, and now that it's not working it's again the fault of the sysadmins. No matter how unmaintainable the system might actually be. Same old, same old in other words. Nobody ever looks at or blames the wonks who got promoted for delivering an unmaintainable system of course.

0
0
Anonymous Coward

Re: Same old, same old

This system has been developed locally by the board, it's got nothing to do with politicians. Every health board is doing the same, developing something similar but as usual there's no one system that fits all due to regional demands being different, but they'll all still interconnect when up and running where required.

Political interference in the NHS is rife but it's not the cause of this, although I did find it funny that the muppet Scottish Health Minister said that boards would review their backups etc. I think you'll find most do that very regularly anyway.. usual minister talking about something he clearly knows nothing about.

This will boil down to one of two things:

1. Incompetent management or staff member.

2. Lack of oversight of change management.

0
1
WTF?

Doesn't sound like a major problem or they've got a decent fallback system in place

"NHS Greater Glasgow and Clyde (GGC) had to cancel 288 outpatient appointments yesterday along with four planned inpatient procedures, 23 day cases and 40 chemotherapy treatments when IT systems crashed."

With such a huge scale of operations across Glasgow, the quoted loss of activity is tiny. I'll bet the usual daily DNAs plus cancellations due to staff/theatre non-availability is ten times greater.....

0
0

Re: Doesn't sound like a major problem or they've got a decent fallback system in place

Bang on the mark. There are very few parts of the organisation that have a critical reliance on IT, mostly outpatient clinics where records, test results and referrals from GP's are pulled from various internal and external sytems via the Clinical Portal.

0
1
Anonymous Coward

Re: Doesn't sound like a major problem or they've got a decent fallback system in place

So....is it the Citrix platform which fell over?

0
0

Page:

This topic is closed for new posts.