back to article UK banking was struck by one IT fail every day for most of 2018

The UK banking sector was hit by IT outages on a daily basis in the last nine months of 2018, with 302 reported TITSUPs according to consumer group Which?. Financial institutions have been required to report major operational or security incidents to the sectoral watchdog since April 2018, and Which? used this data to assess …

  1. Zog_but_not_the_first
    Boffin

    IT Failure?

    Or third party attack?

    1. Anonymous Coward
      Anonymous Coward

      Re: IT Failure?

      Insightful comment or wild, unsupported speculation that provides no real value?

      Feel free to provide support to your argument.

    2. Version 1.0 Silver badge

      Re: IT Failure?

      It was just "the cloud" ... face it, the main function of cloud computing is to generate income to the "cloud providers" reliability is just a competitive feature that aids sales. If you move to the cloud then you're moving all support and reliability to the cloud ... and we all know what clouds do, don't we?

      1. Anonymous Coward
        Anonymous Coward

        Re: IT Failure?

        "It was just "the cloud" "

        Care to explain further? I'm not aware of any of the major banks where failures were broken out using "the cloud" as most IT people would describe it (i.e. using shared platforms managed by a third party with minimal control over the hardware or who can share the resources).

        Unless you're referring to customers accessing in-house or third-party dedicated data centers contracted by banks over the Internet?

      2. Anonymous Coward
        Anonymous Coward

        Re: IT Failure?

        That's why banks generally are not doing this for core systems, or if they are they're doing it which very fierce contracts around it.

      3. Allan George Dyer
        Trollface

        Re: IT Failure?

        @Version 1.0 :

        'face it, the main function of cloud computingBanking* is to generate income to the "cloud providersBankers"'

        FTFY.

        * Feel free to replace with your currently most-despised business sector in any laissez faire economy.

    3. LucreLout

      Re: IT Failure?

      Or third party attack?

      Trust me, all of the banks on that list have technology management and a great many staffers so thoroughly incompetent that they would require no outside assitance for such fuckups. Mines no different.

      Banks do technology bigger, not better, because they have the same afflictions as the rest of the world - too many MBAs, far too many technology managers that don't understand technology, and no realisation whatsoever that in terms of having a future, they must be a technology company first and a bank second.

      1. Anonymous Coward
        Boffin

        Re: IT Failure?

        They also have a problem which many technology companies don't have: legacy. Systems whose direct ancestry (not the hardware, but the platform) dates back to the 1970s are common, and those systems must work. Compare that, say, to Google, who might have platforms dating back to 2000 or so, and for almost all of whose systems occasional failure is fine (who even knows if a search returns the 'wrong' result, or if gmail is offline for a few people for a little while?). And Google are also completely happy to just burn their users if they decide something is boring & have done so repeatedly: a bank can't just turn off the system that supports some saving account they once offered, perhaps for a generation after they closed it to new accounts.

        Banking is actually a hard IT problem. The banks are not doing a great job of solving it, but that doesn't make it easy to solve.

    4. Shady
      FAIL

      Re: IT Failure?

      Hanlon's razor almost definitely applies here

  2. alain williams Silver badge

    Can't spend too much getting IT right ...

    as there won't be enough dosh left for the top brass bonus pot.

    1. Crisp

      Re: Can't spend too much getting IT right ...

      Why would you want to spend money on getting IT right?

      It's not going to cost the company that much in lost business is it?

      btw, how much is Talk Talk worth now?

  3. Shadow Systems

    Cashless society?

    Hastened along by the banks absconding with our money & not letting us have any of it? I think I like the mattress idea, at least that way I can get to my money. Ok, so I won't have an ATM card to swipe everywhere, but then neither will I be beholden to some idiot financial institution that thinks "account security" is an evil to be avoided at all costs. (Gives TheFinger to my old bank that had their picture taken to go beside the definition of "cockwomble" in the dictionary.)

    1. Anonymous Coward
      Anonymous Coward

      Re: Cashless society?

      If your bank is absconding with your money and not letting you have it then you should probably be taking legal advice.

      1. Shadow Systems

        Re: Cashless society?

        I meant situations like the bank going TITSUP, their ATM's crashing, or any other situation where the bank is technicly unable to give you money.

        If the bank "just doesn't feel like it" then that's when I'll drive my car through the front windows, plow through the lobby, & park it on the chest of the cockwomble that refused to give me my money. Sure I could just call the police & let the law take care of it, but where's the fun in that? =-)p

        1. Anonymous Coward
          Terminator

          Re: Cashless society?

          Well, banks do a huge amount of work to make sure that doesn't happen very often. Sometimes it does, but it's actually pretty rare, despite this article: not very many of these serious incidents will have involved ATMs being down or inability to handle transactions (some will, just not very many). It seems less rare than it is for the same reason people being eaten by sharks seems less rare than it is: every time it happens it's in the news (an average of a little over 7 people were killed by sharks per year over the last 60 years).

      2. LucreLout

        Re: Cashless society?

        If your bank is absconding with your money and not letting you have it then you should probably be taking legal advice.

        It's actually not without precedent. IceSave (remember them) did the same trick when they were going bust - froze out UK and other international customers to preserve enough cash for the Icelandic customers to draw out theirs.

        A cashless society works until you realise it prevents any realistic run on the banking system without a massive gold spike also accompanying it. The problem with that is the bid/offer spread on retail level physical gold (sovs, krugs etc).

        1. Anonymous Coward
          Anonymous Coward

          Re: Cashless society?

          IceSave is kind of the canonical example of why people should bank with banks covered by the FSCS scheme.

    2. ecofeco Silver badge

      Re: Cashless society?

      I recently got screwed by an on-line payment system without recourse.

      Fuck them all.

      Don't bother with blaming the victim, the details are not what you think they are and there truly is no recourse for me.

      1. Anonymous Coward
        Pirate

        Re: Cashless society?

        Well, either you entered into whatever arrangement it was knowing that this could happen (or you just did not read the fine print) or they have broken their contract with you / the law in which case you have recourse. Since you state you have no recourse I must assume the former.

  4. Anonymous Coward
    Anonymous Coward

    Mattress

    How do I store a collection of bits under my mattress? I don't think the bank will send me my money on an SD card.

    1. Anonymous Coward
      Anonymous Coward

      Re: Mattress

      I find vacuum sealing the bits avoids most of the issues with smell and leakage, although I find storing the bits away from people avoids most of the issues with being caught.

      Storing them under your mattress just sounds uncomfortable.

      1. TimMaher Silver badge

        Re: Mattress

        Could always use the freezer?

  5. chivo243 Silver badge
    Facepalm

    it is time!

    to keep a couple of weeks worth of cash for expenses on hand, just not on my person 24\7. I guess the mattress is as good as coffee cans in the garden ;-}

    1. Rich 11

      Re: it is time!

      I don't bury money in coffee cans in the garden anymore, not since last winter when the squirrels dug the can up and spent the money on nuts.

    2. steviebuk Silver badge

      Re: it is time!

      Just don't let anyone know or you'll end up in the dock like Ken Dodd :) although there is nothing illegal about hiding your money in the wardrobe or in your mattress.

    3. Will Godfrey Silver badge
      Happy

      Re: it is time!

      What do you mean? I never stopped keeping an (undisclosed) sum of money in mixed coin and notes in several (undefined) locations, and have found it very useful on many occasions over the years.

      P.S. I also keep an eye on currency changes so I don't end up with Old Stock.

      1. Doctor Syntax Silver badge

        Re: it is time!

        "(undefined) locations"

        Quantum money!

        1. Anonymous Coward
          Anonymous Coward

          Re: it is time!

          "Quantum money!"

          No, just left over change that spills out of my pocket when I stumble home from a beer and a curry.

          It does seem "quantum" when I find it though

        2. Crisp

          Re: it's only quantum money

          If you either know how much you've got, or where it is, but not both at the same time.

        3. tapemonkey

          Re: it is time!

          In other words you have money. Then when you look for it its gone. Sounds like a normal day in any home with teenagers

          1. quxinot

            Re: it is time!

            Or my wife.

  6. chelonautical

    The meaning of "major incident"

    Interesting data, but I don't find these raw figures very helpful on their own. As the correspondent noted, there's no mention of the duration of the incident. Also, I don't get any sense of the severity of the impact as there's no detail about which bank functions were affected. Some of the things that banks do are more critical or time-sensitive than others.

    As an example, the inability to process outgoing debit card payments for even a couple of minutes could be extremely inconvenient (not to mention embarrassing) for the customer. By contrast, if the system that handles my outgoing standing orders is down for (say) 20 hours but eventually manages to send out a payment slightly late on the same day it was already scheduled to be paid then I probably won't even notice that. The first outage is a real nuisance, the second far less so - although still serious from the bank's perspective, of course.

    I was curious as to what "major incident" means so I went digging... The relevant regulation appears to be http://www.legislation.gov.uk/uksi/2017/752/regulation/99/made - it doesn't classify incidents, although regulation 98 says "... the payment service provider must establish and maintain effective incident management procedures, including for the detection and classification of major operational and security incidents". I interpret this to mean that each payment provider can have its own definition of "major incident". In which case comparing the numbers between different banks won't necessarily yield an apples-to-apples comparison. A more honest bank (if such a thing exists!) would look worse than its competitors, so without strong oversight of the criteria used, the likely effect of this kind of publicity is a race to the bottom to define "major incident" as narrowly as possible, thereby defeating the purpose of the regulations in highlighting such problems.

    It's a pity because in principle I welcome greater transparency in this area, just not sure it will work as intended.

    1. katrinab Silver badge

      Re: The meaning of "major incident"

      Also, TSB was one very major incident which lasted about 3 months, whereas Barclays sometimes goes down at the end of the month when most employers are paying their salaries and therefore it has the highest volume of transactions, and tends to be back up the following day.

      1. Rich 11

        Re: The meaning of "major incident"

        It's OK, Barclays have a cunning plan to fix the load problem by reducing the number of people being paid at the end of every month.

      2. Anonymous Coward
        Anonymous Coward

        Re: The meaning of "major incident"

        "Also, TSB was one very major incident which lasted about 3 months"

        According to the BBC earlier today (https://www.bbc.co.uk/programmes/m0002z8l), TSB have now addressed issues for 97% of the 200,000 affected users.

        So just another 6000 peoples accounts to go...

    2. commonsense

      Re: The meaning of "major incident"

      Getting an organisation to be consistent within its own four walls on what constitutes a major incident is difficult enough, never mind having consistency across an industry.

      In some places, if you do your job right by documenting SLAs carefully and therefore report more incidents as major, and you create a rod for your own back because you create the impression there's something inherently wrong within your organisation (and there probably is).

      I don't think you can read too much into the number of incidents reported. Knowing what banks are like, it could be only some of them are playing ball, and even then only their particular variant of ball game.

    3. Adrian 4

      Re: The meaning of "major incident"

      "A more honest bank (if such a thing exists!)"

      Don't sweat the edge cases.

  7. Doctor Syntax Silver badge

    TITSUP = Titisup Is Typical System's Usual Performance.

    1. DCFusor

      Ah, recursive acronym. What's old is GNU again?

  8. Anonymous Coward
    Anonymous Coward

    I notice RBS, NatWest and Ulster reported separately, for the total should they be added up or were NatWest and Ulster outages subsidiaries of the RBS group ones?

    1. Anonymous Coward
      Anonymous Coward

      That's a good point. Pretty sure that NW and RBS share infrastructure more-or-less entirely, Ulster I think so although not quite so sure.

    2. katrinab Silver badge

      They have separate banking licences, and I assume the FCA reports per banking licence even if the same issue affected multiple banks.

    3. paulf

      Similarly Lloyds and Halifax/Bank of Scotland share the same Lloyds platform but have reported separately, perhaps because they have separate banking licenses. Not sure I agree with this as it depends on how segregated things are on the platform in question. Is it a platform problem that only affects selected brands/licenses using that platform or does it affect all brands/licenses using that platform.

  9. Stoneshop
    Trollface

    Duration

    The surprise result will be IT meltdown bank TSB, which reported just 16 incidents during the final three-quarters despite its chaotic year.

    So, 18 days on average (El Reg reporting on TitSupBank gives me the impression they've been out the entire period, more or less)

  10. Anonymous Coward
    Anonymous Coward

    is this a problem on this scale elsewhere in the world?

    Is it just UK banking that's this bad, or is it a problem on this scale elsewhere too?

    And if it's better elsewhere, what can Johnny Foreigner learn from the UK?

    (Obviously there's no possibility that England's Masters of the Universe (Retail Banking subdivision) could learn from anyone elsewhere).

  11. Stevie

    Bah!

    UK banking was struck by one IT fail every day for most of 2018

    Good news: not all down to TSB.

  12. Anonymous Coward
    Anonymous Coward

    We've just had one incident . . .

    PRI 1: "Everything is fucked"

    Incident is ongoing, with no estimated time to resolution.

    1. FozzyBear
      Devil

      Re: We've just had one incident . . .

      Same here. The ticket was been open for more than 5 years now. ( originally created to test the new ticket system). We leave it open and omit it from the SLA reports. We assign it in the second week, whenever a new person starts.

      The range of reactions has been amazing, from WTF! to chicken little

  13. Anonymous Coward
    Anonymous Coward

    However, Which? pointed out that the data doesn't show how long services were down for.

    So an apples vs pears comparison then?!?

    Didn't data analysts understand the concept of Risk Priority Numbers? In this case, the product of Severity, Frequency and Duration scores.

    Or is the objective to draw a conclusion not supported by the data?

  14. ecofeco Silver badge

    It's epidemic

    Major systems are breaking everywhere every day.

    No, this is NOT normal. That anyone would think this is acceptable and normal shows how bad it has become. It also points to a serious coming catastrophic event in the near future.

    Systematic failures are not a sign of normal operations. Ever.

    1. Bronek Kozicki

      Re: It's epidemic

      They are normal in the UK bank sector :-( And they will stay normal until the regulators force banks to improve, rapidly.

      No, really.

      Basically what we are seeing is the inability of bank management to manage the technological risks properly.

  15. Bronek Kozicki

    "The cloud"

    Funny thing, there is one bank on the list which is running its systems exclusively "in the cloud". It is Starling, and the number of failures is exactly zero.

  16. Anonymous Coward
    Anonymous Coward

    pay peanuts: get monkeys

    Who ever would have thought that replacing loyal, well paid career IT staff with decades of experience and deep understanding of their systems with whoever can be had globally at the lowest cost with no job security would end in tears? I'm not complaining having left a long time ago with big redundancy pay-off and good final salary pension.

  17. Herring`

    It's interesting* that it's only in Monopoly that you get a bank error in your favour.

    *I lied

  18. StuntMisanthrope

    bankofeastindia.com

    It’s cheaper, cuts out the cost of payroll and is government backed. Works out of hours when required as opposed to your paid-time and weekends.

    I believe they have a plan to outsource cash machines to localised third parties as there appears to be about a 50% decline in availability. #madraspoints

    1. StuntMisanthrope

      Re: bankofeastindia.com

      With roboATM’s on wheels that auto-fill and swarm. #monkeybusiness

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon