back to article 'Powerful blast' at Glasgow City Council data centre prompts IT meltdown

The catastrophic service outage at Glasgow City Council's data centre, caused after its IT systems servers were taken down by a fire suppressant accidentally going off, is continuing to cause widespread havoc for staff and the public. The embarrassing blunder was caused by a faulty air conditioning unit setting off its fire …

Page:

  1. dervheid
    Trollface

    Must have been...

    The SNP.

    Get blamed for everything else

    1. Anonymous Coward
      Anonymous Coward

      Re: Must have been...

      if the hat fits..... ;)

      1. Halfmad

        Re: Must have been...

        Hat doesn't fit, bloody SNP can't even get one the right size.

    2. Anonymous Coward
      Anonymous Coward

      Re: Must have been...

      #SNPBad....even if it is a labour council

    3. Velv
      Headmaster

      Re: Must have been...

      In exactly the same vein as Godwin's law, Rubislaw is the Scottish equivalent where everything is reduced to the fault of the UK Government at Westminster.

      Clearly Rubislaw has kicked in pretty quickly on this one...

      1. This post has been deleted by its author

    4. unwarranted triumphalism

      Re: Must have been...

      They found something else to sabotage then.

    5. Anonymous Coward
      Anonymous Coward

      Re: Must have been...

      I like how the picture on the article is a f****g kilt. Sod all to do with the story beyond the fact it's in Scotland, they couldn't even be arsed bringing out the Glaswegian stereotypes. Scotland = Shortbread Tin Highlander.

      Anyway, it turns out that Sheffield Council are having some problems with their IT systems. Here's a picture of what that might look like. A spokesman said "Lor luv a duck, I know I'm not even the right faacking stereotype, but it's all daaaarn saaarf, innit! Root-te-toot, apples and pears, etc."

      1. Anonymous Coward
        Anonymous Coward

        Re: Must have been...

        Pictures? ... fortunately the "banner" pics in all reg articles follow teh same URL pattern so its simple to use a custom AdBlock rule to hide them - been doing this since they arrived and can't say I've ever felt I'm missing anything. Also added another rule to hide those stupid "badges" and along with all the other stuff that gets blocked makes ElReg a much more pleasant read!

      2. Dieter Haussmann

        Re: Must have been...

        I see you're new here.

      3. Anonymous Coward
        Anonymous Coward

        Re: Must have been...

        "I like how the picture on the article is a f****g kilt. Sod all to do with the story beyond the fact it's in Scotland, they couldn't even be arsed bringing out the Glaswegian stereotypes"

        I agree completely...

        This picture is far more appropriate to Glasgow and its people

        1. Mpeler
          Pint

          Re: Must have been...

          Which explains the town's name:

          Na wheah de-ad da galass go?

          [grabs me dram and coat - weel I'm aff]

    6. TheVogon

      Re: Must have been...

      "powerful blast of gas"

      Probably not unusual with a Scottish diet....Probably it was really caused by someone plugging a family size ActiFry into the UPS supply!

  2. Roger Kynaston
    Mushroom

    Business Continuity

    I wonder if they tried to invoke an underfunded and fundamentally unworkable business continuity plan.

    1. Halfmad

      Re: Business Continuity

      I wonder if they even had one.

    2. This post has been deleted by its author

  3. thedroog

    They spent their budget on deep fried marsd bars...

    1. Turtle

      @thedroog

      "They spent their budget on deep fried mars bars..."

      You say that as though it's a bad thing.

    2. DJV Silver badge

      @thedroog

      "marsd"

      Is that some sort of Unix daemon?

  4. Anonymous Coward
    Joke

    Resiliency testing

    A little extreme but their testers can go home feeling warm inside now that they've found a single point of failure.

  5. thedroog

    Would that single point be their DR plan?

  6. Anonymous Coward
    Anonymous Coward

    Well bugger me, talk about loss of 'essential' services. How are councillors going to claim expenses now? Also, where will people continue to be able to, umm...

    What do councils do again apart from collect bins and claim expenses?

    1. Anonymous Coward
      Anonymous Coward

      Sometimes they like to fuck up their filing systems and then spend a decade pursuing whoever pointed out the mistake in order to extract large sums of money, at the same time as they're breaching the data protection act and tossing around threats of arrest and imprisonment.

      All because I pointed out that no, I'm not my dad, please correct your records...

    2. localzuk Silver badge

      Maintain roads. Schools. Social care. Pre-school education. Children's/Family services. Youth centers. Youth justice (eg. secure accomodation), street lighting, Social housing. Dealing with homelessness. Museums and galleries. Recreation areas (parks, leisure centres, sports facilities, playgrounds etc...), tourism information and promotion, libraries, cemetaries/cremation, trading standards and community safety, environmental health (food safety, pest control, pollution control), licensing (alcohol licensing, taxis, events), garbage collection, planning control, regeneration, fire services, coroners, registration of births, deaths and marriages, and election admin...

      So yeah, what do they do for us hey?

      1. NeilPost Silver badge

        That's why they have so many staff, capable managers and get fuck loads of taxpayers money.

      2. TitterYeNot

        "So yeah, what do they do for us hey?"

        Yeah, bloody Romans...

        ROMANES EUNT DOMUS! ROMANI ITE DOMUM!

      3. Prst. V.Jeltz Silver badge

        "So yeah, what do they do for us hey?"

        dont need *any* of that stuff, except collect bins

    3. Matt Quinn

      Whaddaya mean "collect bins"?

  7. Anonymous Coward
    Facepalm

    Jeez, what kind of gas-based fire suppression system shakes buildings and seriously damages equipment?

    1. Mystereed

      Isn't that how Red Adair used to put out oil well fires - big explosion to snuff the flames?

    2. Diodelogic

      If a burrito-fueled irruption can do it, why can't a fire suppression system?

    3. Mr Dogshit
      Boffin

      If the amount of kit in your data centre changes significantly, you need to have the fire suppression people recalculate the amount of gas needed. Too little gas, and the fire isn't extinguished.

      Too much gas... and you can blow the walls out.

      1. Anonymous Coward
        Anonymous Coward

        If the amount of kit in your data centre changes significantly

        Some clarification needed. Most IT kit is largely air by volume. And when the system goes off, presumably the air in the data centre has to be displaced to somewhere, otherwise the pressure must build up.

        What kind of rate of pressure rise are we looking at? How is it vented?

        1. Anonymous Coward
          Anonymous Coward

          Re: If the amount of kit in your data centre changes significantly

          Okay, imagine a room three metres wide, three metres long, three metres high.

          That's nine cubic metres by my reckoning.

          The room contains three cubic metres of kit. The fire company has calculated the quantity of FM-200 or whatever it is to fill the remaining six cubic metres. If you add more kit and the gas goes off, it's got to go somewhere. So either the roof pops off or the walls shift.

          That's GCSE physics surely. That's my understanding as a humble data centre BOFH not a million miles from Glasgow, I'm not a gaseous fire suppression engineer.

          P.S. The SNP will most likely blame "Westminster" and/or David Cameron. Still, we've got road signs in Gaelic, and that's really important.

          1. theblackhand
            Flame

            Re: If the amount of kit in your data centre changes significantly

            All this talk about faulty fire suppression equipment, but no body seems to be mentioning how there was NO fire damage to any of the equipment.

            Impressive no?

            Where is the suppressed fire icon when you need it?

            1. Anonymous Coward
              Anonymous Coward

              Re: If the amount of kit in your data centre changes significantly

              Not quite sure what happened. And what do they mean by a faulty AC unit? Was it faulty as in going to catch fire!? If there's a VESDA (Very Early Smoke Detection Apparatus) system then that would be set off by the AC if it was producing prefire vapours (that's the whole point of VESDA) it detects "fires" before they turn in to fires! But normally you wouldn't want this to set off your suppression system, initially you'd want it to display an alarm on the fire panel so it could be investigated, not trigger the house alarm and discharge your gas! Or was it faulty and something happened like it produced a load of dust or something else that could also have been detect by the VESDA? This is why its important to remember to disconnect your suppression system BEFORE you let anyone in the DC to service kit etc! But whatever it shouldn't have set the gas off and the release of gas shouldn't have caused the damage that it did!

            2. NeilPost Silver badge

              Re: If the amount of kit in your data centre changes significantly

              The story mentions failed AC equipment, but no real info on the exact nature. If my experience, AC failure units pissing water down the walls of a server room, and it cooking at 70C until the servers auto-power down is the main symptoms you get.

          2. Commswonk

            Re: If the amount of kit in your data centre changes significantly

            Okay, imagine a room three metres wide, three metres long, three metres high.

            That's nine cubic metres by my reckoning.

            Eh? 27 cu. m, Shirley?

            That's GCSE physics mathematics surely.

            1. Anonymous Coward
              Anonymous Coward

              Re: If the amount of kit in your data centre changes significantly

              Yeah, that. Good point well made.

              See, I'd make a lousy fire suppression guy.

              1. Sir Alien

                Re: If the amount of kit in your data centre changes significantly

                Can't really calculate it purely by kit space. The air in the datacentre also needs displacing. The rooms should really have a one way pop vent to relieve excess pressure but not let air back in.

                1. Anonymous Coward
                  Anonymous Coward

                  Re: If the amount of kit in your data centre changes significantly

                  yep normally you have some louvres or other form of vent to vent the air in the DC out to somewhere else. Our DC isn't massive so it vents in to the surrounding corridors

                2. Prst. V.Jeltz Silver badge

                  Re: If the amount of kit in your data centre changes significantly

                  I dont see why you need to calculate area and kit volume etc. I would just

                  - position inert gas dispensers at one end of room

                  - put hole at other end

                  -in event of fire blast inert gas into room - allowing air to escape through hole

                  - when sensors reveal air gone and room full of inert gas, switch off.

            2. John Brown (no body) Silver badge

              Re: If the amount of kit in your data centre changes significantly

              That's GCSE primary school physics mathematics sums surely.

              FTFY :-)

              1. RedCardinal

                Re: If the amount of kit in your data centre changes significantly

                >>That's GCSE primary school physics mathematics sums surely.

                Lol not in today's British education system. You'll be lucky if it's even GSCE :P

                1. Matt Quinn

                  Re: If the amount of kit in your data centre changes significantly

                  Ummm... There is actually no such thing as a "British education system". - Never has been... No GCSE up here either; unless taken as a 'foreign' exam.

            3. Anonymous Coward
              Anonymous Coward

              Re: If the amount of kit in your data centre changes significantly

              @ Commswonk

              More importantly, how did you know his name was Shirley? You must work for the NSA!

            4. channel extended

              Re: If the amount of kit in your data centre changes significantly

              Also if you have 24 cu m. of air and you double it with a gas, depending on what type, you get 2 atmo of pressure or 28.8 lb/sqr inch. while this might blow out a loose window pane it should not be enough to blow out walls. In a well designed data centre.

              Yes hurricaines and tornados do blow down building using less pressure, however they have wind shear and wind velocity to help.

          3. PNGuinn
            Joke

            Re: If the amount of kit in your data centre changes significantly

            "Okay, imagine a room three metres wide, three metres long, three metres high."

            3x3=9x3=27.

            Bloody Gaelic arithmetic.

            Clearly the fault DC and Westminster.

          4. localzuk Silver badge

            Re: If the amount of kit in your data centre changes significantly

            Gases can compress pretty well, and if your walls blow out due to a small change in the contents of a room, then those walls were made of paper.

            The building shaking is likely as a consequence of the fixed equipment venting all that gas in a matter of a couple of seconds. There's gonna be an equal and opposite reaction to the gas being deployed somehow...

          5. S4qFBxkFFg

            Re: If the amount of kit in your data centre changes significantly

            P.S. The SNP will most likely blame "Westminster" and/or David Cameron. Still, we've got road signs in Gaelic, and that's really important.

            No need, Glasgow City Council is Labour run (and has been for years).

            Also, the Gaelic tends to only get added when the signs are getting replaced anyway.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like