back to article Scottish NHS bosses say soz after 2-day IT ballsup scrubs 700 appointments

A Scottish health board has issued a grovelling apology to the hundreds of patients whose appointments were cancelled due to an embarrassing system error. Although NHS Greater Glasgow and Clyde (GGC) has now fixed the "unprecedented" fault, some 709 appointments bit the dust. The issue started on Tuesday and was only resolved …

COMMENTS

This topic is closed for new posts.
  1. Pen-y-gors

    Please speak English (or Scots)...

    "Although 709 patient episodes have had to be postponed..."

    WTF is a 'patient episode'? Something to do with Holby City?

    1. S4qFBxkFFg
      Pint

      Re: Please speak English (or Scots) [or Latin]...

      "It would appear that no data appears to have been lost."

      "The servers [are] now reloading, no data has been lost and 50 per cent of users now have access to the system with hopes the remainder would have access by evening."

      Despite what the newer style guides might say, treating "data" as a singular noun still causes my brain to soft-reset every time I read/hear it.

      Here's to the Friday-beer method of desensitisation.

    2. The Mole

      Re: Please speak English (or Scots)...

      The patient episode was bad enough, but 're-appointed' and 're-appointment'? If they all speak like that perhaps England should disown the entire country?

      The patient hasn't been appointed to a position, they have been re-booked and given a new appointment.

      1. roblightbody

        Re: Please speak English (or Scots)...

        Bad grammer happens all over the UK....

        England cannot disown Scotland, as it does not own it. It could disassociate itself perhaps.

        1. Anonymous Coward
          Anonymous Coward

          Re: Please speak English (or Scots)...

          Bad spelling too, apparently.

    3. Dr Who

      Re: Please speak English (or Scots)...

      An episode is defined in the dictionary as "a single event or group of related events". In the NHS the word is routinely used to describe everything that goes into your period of treatment. This may start with an operation but also include a drugs regime, a stay on a ward for recovery, and follow up visits as an outpatient. It's actually a very appropriate word to use in this context.

  2. The Original Steve

    Bullshit

    Anyone saying that AD was corrupt, but also calls it "a router system" is clearly clueless.

    Less said about taking 2 days to restore a fecking LDAP database the better....

    1. smudge
      Thumb Down

      Re: Bullshit

      This was the Chief Exec speaking to a non-tech audience. He is quoted as saying "...Active Directory, a router system which recognises users and allows individual access to our clinical and administrative support systems".

      Now you and I wouldn't have used the word "router". But from a non-tech person to a non-tech audience, it's fine. AD controls what you get access to.

      Presumably you don't often communicate with non-IT specialists?

      1. Anonymous Coward
        Anonymous Coward

        Re: Bullshit

        What would be annoying for him is if he came across one of those annoying people with just a little bit of advanced knowledge, but not that much and they think they are f$ck1ng experts on everythng IT.

      2. Anonymous Coward
        Anonymous Coward

        Re: Bullshit

        The way it reads to me is that they're using something on top of Active Directory to grant access to various databases.However, Unless they publicly release an analysis of the root cause we'll never know

    2. Anonymous Coward
      Anonymous Coward

      lost in translation

      I'm sure the techs who explained it to the person who then relayed the message to the individual who wrote the script for this announcement mentioned "AD", "router", "system", "corrupt" and "users" but not necessarily in the same order that it came out.

      1. Captain Scarlet

        Re: lost in translation

        Or maybe after explaining it for the 50th time the person being grilled whilst trying to fix it got fed up and thought screw it I'll say stuf he will recognise so they go away.

    3. R 11

      Re: Bullshit

      Likewise, I have no idea why it took 2 days to restore. However, taking a 'less said' approach is foolish. This was getting a lot of publicity - I think you can safely say there would be a lot of folk from whatever software and hardware companies that were involved were working on this.

      Personally, I'll never say this couldn't happen to me. Crikey, even Google had to resort to restoring lost gmail messages from tape a while ago. The unexpected can happen to anyone.

      I'll look forward to seeing the postmortem and finding out what went wrong. Perhaps someone made a stupid mistake. Perhaps, there will be a lesson or a warning for us all.

  3. smudge
    Headmaster

    National holiday?

    "...after a national holiday in some parts of Scotland."

    I do realise that there's only one icon that I can attach to this comment :)

    1. K
      Thumb Up

      Re: National holiday?

      Have +1, help counter the idiot who downvoted a valid point

      1. Intractable Potsherd

        Re: National holiday?

        I wasn't the downvoter, by the way.

    2. Intractable Potsherd

      Re: National holiday?

      You are clearly not familiar with the vagaries of holidays up here. For reasons I have yet to fathom, Dundee does not have several of its "national"* holidays at the same time as the rest of Scotland. The one mentioned here is actually taken in Dundee next Monday. The only way I know when these holidays are is to keep an eye on the front door of the local GP, since they inform patients when they will be closed several weeks in advance!

      * To Scotland, and better referred to as "public holidays"

      1. smudge
        Happy

        Re: National holiday?

        So if only Tammy Troot* promised to unify holidays throughout Scotland, the referendum would be a walkover for him?

        *Nickname for Scotland's First Minister, based on his surname and a children's book character.

      2. K

        Re: National holiday?

        heres a thought, RTFM.. or in this case a dictionary -

        "relating to or characteristic of a nation; common to a whole nation"

        Since that is clearly not the f*cking case (as the article author clearly states), then it cannot be "national"..

        holy shed.. pass me an inhaler..

  4. Anonymous Coward
    Anonymous Coward

    It's clear for the quotations in the story that the people talking know absolutely sod all about IT. "Server are reloading", almost as good as yesterdays "running diagnostics" quote on servers which had apparently been fixed, bit late with those diagnostics guys.

    Like the Health Minister asking all health boards to check backups and servers.. really like they don't already do that, this is an entirely separate entity, why not get every health board and local authority in the UK to do it - it'd be just as much bloody use.

  5. Anonymous Coward
    Anonymous Coward

    a Router system eh!? Sounds like the kind of thing our head of operations would come out with he's a totally clueless t*at that spouts rubbish about things he knows sod all about.

    1. theblackhand

      In the chief executives defence...

      I've had the misfortune to be involved in health records management systems in the past and I suspect the router that he was referring to was a patient records router rather than a router for network packets.

      The HL7 routers we used involved large amounts of pixie dust and magic to work. Just because a setting was clearly incorrect, under no circumstances should it ever be changed unless every system using that setting is double checked and a Magic Eight ball returns 5 "Definitely"'s in a row.

      My guess is that the AD corruption was an incorrect or deleted DNS entry and that a router stopped passing records between systems because a router could no longer find a destination system.

      Pure speculation....

  6. Bluenose

    “Although 709 patient episodes have had to be postponed,

    But they will be broadcast on BBC Alba and STV at a future date (probably after Independence Vote)

  7. Anonymous Coward
    Anonymous Coward

    This is what happens when you contract the support and management of your IT to the lowest bidder (no matter what shore they are on).

    You don't pay the money just for when its working. You pay the money to get it fixed right quickly first time when it goes wrong. Aeroplanes don't "need" pilots any more - except when the planes go wrong, at which point the pilot REALLY earns his pay.

  8. Anonymous Coward
    Anonymous Coward

    Sounds like a similar problem i just had to help with

    I've just finished an emergency contracts for a Dental holdings company that sounds like a very similar problem which only affected Windows 2008 boxes from what I understand (I was only there to bring servers back online and not fault diagnostics) the server was receiving too many request from an application so rebooted on restarting into windows they got stuck in a loop of rebooting not being able to get to a log in screen, the entire domain had to be rebuilt and a known good AD structure transferred to it with each affected server had to be physically brough in to Active Directory restore mode then taken off and readded to the domain to correct the issue with the servers. It affected around 290 servers.

  9. Andy Gates

    I'd be impressed if all their systems are AD-integrated that tightly. There's commonly lots of legacy stuff around (and lots of pixie dust).

    If our experience with single-sign-on applications is anything to go by, I'd lay odds on that layer failing over AD. AD is pretty stable; SSO is pixie-dust, and after the first password change cycle, only it knows the passwords to linked systems. If it were to barf spectacularly, they'd be up a well-known creek.

  10. localzuk Silver badge

    Whut?

    I gotta know - how did it take them 2 days to restore from backup?!

  11. John Smith 19 Gold badge
    Unhappy

    So what happend and *why*

    Because it looks like a total f**k up to me.

  12. Anonymous Coward
    Anonymous Coward

    Active Director© corruption ..

    "Mr Calderwood said 99% of all IT networks across the UK are built on the relevant program called Microsoft active directory and it was the first time the health board had experienced a fault in more than 10 years working with Microsoft.

    Mr Calderwood added: "Microsoft itself says it has never had such a fault reported across many years in all of its business areas."

    http://www.heraldscotland.com/news/home-news/deadline-set-for-cancer-therapy-after-it-glitch.22325494

    https://www.google.co.uk/search?as_q=corruption&as_epq=active+directory

This topic is closed for new posts.

Other stories you might like