back to article Who will fix our Internal Banking Mess? TSB hires IBM amid online banking woes

In the midst of its six-day online banking meltdown, which is showing no sign of letting up, TSB has hauled in the systems integration big gun or as it is otherwise known, IBM. The British bank confirmed the signing of Big Blue this morning as part of its rather uncomfortably timed first quarter financial results for 2018, in …

Page:

        1. werdsmith Silver badge

          Re: not a customer

          I was a customer of TSB for many years/decades.

          Then after the merger I became a customer of LloydsTSB.

          Then when TSB was sold off I emerged as an involuntary customer of Lloyds.

          Dodged a bullet.

        2. d3vy

          Re: not a customer

          "I'm a TSB customer and as far as I can see, my direct debits and standing orders have worked just fine since Monday morning."

          They have been "Core services" were up and running at the weekend... but no one can access their accounts.

          I have a few accounts with TSB one which I use day to day and one for the bills. At the end of every month I work out the next months outgoings and transfer them into the bill account which I then dont touch - this means that no matter what happens during the month the bills are all covered.

          The problem now is that as its the end of the month that bill account is empty, I cant put any money in it because the account with the money in is a TSB business account (and the internet banking for the business accounts is in a worse state than the normal accounts).

          So shortly my mortgage and a host of other payments will come out of my bill account - this will do one of two things (Maybe both)

          1. Put me WELL over my overdraft

          2. DD will get declined and go unpaid.

    1. d3vy

      Re: not a customer

      "sounds like a complex undertaking done for good reasons."

      Well, You're right to an extent.

      It IS a complex problem and they do need to do it.

      "Customers should just be patient and not get agitated."

      I wasn't agitated until at 7pm on Sunday (AFTER the switch over was meant to be complete) when my debit card stopped working (Its still not working now).

      ATMs also seem sporadic - Though to be fair I've only tried once.

      I can tell you now that 6 days on I am more than agitated I have two personal accounts one which is used just for bills and by this time of the month will be empty. The other has money in it but I cant get to it.

      I also have three business accounts which I cant get to to pay wages out.. or check that Ive been paid by clients..

      So my only option now is to go into a branch and HOPE that they can transfer the funds for me - from what I have read online this is a bit hit or miss at the moment.

    2. Twanky

      Re: not a customer

      "sounds like a complex undertaking done for good reasons. Customers should just be patient and not get agitated."

      Don't feed the trolls :)

  1. Gordon 10
    Unhappy

    I pity the poor schmoes working on this.

    Now not only do they have a badly planned, poorly managed cut over to repair they are also going to have a bunch of senior muppets from IBM looking over their shoulder second guessing them with the benefit of hindsight.

    I bet somewhere in the depths of TSB's tech dept there are a number of people muttering "told ya so" under their breath who were ignored when they raised issues.

    I don't see what value IBM can add at this point other than making things worse. I get a very strong whiff of "something must be done"

    1. HmmmYes

      Re: I pity the poor schmoes working on this.

      'I bet somewhere in the depths of TSB's tech dept '

      I doubt it,

      Itll just be empty desk, where the people who did work there worked, before being let go.

      1. Anonymous Coward
        Anonymous Coward

        Re: I pity the poor schmoes working on this.

        Nah they will make them fix the problem first then let them go. Even if the problem was caused by management ignoring advice from techies, it'll still be the techies fault ... somehow.

        1. Vincent Ballard

          Re: I pity the poor schmoes working on this.

          Anonymous Coward is spot on. A former colleague of mine is caught up in the middle of this, having joined Sabadell recently in a QA role, and reports that people are pulling seriously illegal amounts of overtime and staying awake with the aid of pills and Bolivian marching powder.

        2. HelpfulJohn

          Re: I pity the poor schmoes working on this.

          "Even if the problem was caused by management ignoring advice from techies, it'll still be the techies fault ... somehow."

          Space Shuttle. O-rings.

          You'd think those who don't know how things work would have seen that one as a Really Big Wake-Up and would now be paying attention to those who do Know Things.

          Or am I expecting too much from Admin and Management Plonks?

  2. BoldMan

    What are IBM going to do? Wave a fucking magic wand?

    So how do they expect a bunch of outsiders to come along and fix everything when their own internal people who built the fucking thing can't get it working quickly?

    Bringing in outsiders always looks good to the mouth-breathing morons up the org chart, but doesn't fix the problem just means that the people who are supposed to be fixing the problems spend pointless time explaining things to the newbies.

    This Peter Pester twat sounds like a complete waste of space.

    1. Doctor Syntax Silver badge

      Re: What are IBM going to do? Wave a fucking magic wand?

      "This Peter Pester twat sounds like a complete waste of space."

      Sounds like the Peter principle at work.

      1. Byz

        Re: What are IBM going to do? Wave a fucking magic wand?

        Except his name is Paul David Pestor.

        Not Peter !!!

        1. Anonymous Coward
          Anonymous Coward

          Re: What are IBM going to do? Wave a fucking magic wand?

          There are a lot of people in this thread who are jumping to some remarkable conclusions.

          A couple of disclaimers - I have no inside info here, and I once worked for IBM (left a few years back).

          Everyone here's pointing the finger at TSB and their new owners here but the reality is most likely that this is all a great example of the evils of lock-in, which can be entirely attributed to Lloyds. There is a lot of IBM presence in that bank and they are very dependent on overpaid contractors to keep it running, along with oodles of offshore bollocks.

          My guess (and it is admittedly a guess, although I hope an educated one), is that TSB are trying to change as little as possible, so are splitting out IBM systems, or at least duplicating them, and someone somewhere has not done the numbers, or more likely (in my vast experience) has been told those numbers (performance metrics, technical necessity etc.) do not match the other numbers (£££) and some serious corners have been cut. I hope the relevant technical people kept their emails on the subject.

          IBM being brought in now is unlikely to be a free market choice; IBM are most likely the best people to fix it because it's IBM technology that's involved. If it's an architectural issue or a sizing issue, as is most likely, and this was not IBM's fault, then IBM aren't going to fix it without getting paid. Support doesn't cover that. The fact that TSB have little choice other than calling IBM is entirely down to lock-in. Of course, they may have had the option to move off IBM here, but probably decided to change as few things as possible to improve chances of success.

          I'll reiterate that this is conjecture on my part, but I have a lot of experience here - being sent halfway round the world to fix customer issues. Of course, I no longer work there (took a package) but there are still people who can probably fix this. It won't be quick though and I can guarantee that the people who do fix it will get zero credit. Which is one of many reasons I no longer work there.

          I am hoping the details do get leaked, but I for one am not pinning this entirely on TSB.

          1. Doctor Syntax Silver badge

            Re: What are IBM going to do? Wave a fucking magic wand?

            "I am hoping the details do get leaked, but I for one am not pinning this entirely on TSB."

            What has to be pinned on TSB is that they took stock of the situation and decided they were in a position to go ahead in a single operation. And then found they weren't.

          2. I ain't Spartacus Gold badge

            Re: What are IBM going to do? Wave a fucking magic wand?

            It's a bit of an EU cock-up in my book too. RBS have apparently wasted spent a couple of billion trying to sell several hundred branches to comply with competition rules - but nobody in the end wanted to buy them. So they're now spending billions more trying to spin them off as Williams and Glyn. That means setting up a head office, management, IT, and a cash pile/reserves.

            This TSB thing is a similar exercise in regulatory box-ticking. That's doing nobody much good. How much have we really gained as customers attracting another Spanish bank into the retail market, as a minor player with just a few hundred branches?

            We had state aid and shotgun mergers during the financial crisis and that broke competition rules. But they really should have ignored them. At least until a later date. There's a reason that when banks buy other banks they tend to leave them on the old systems for years, and only slowly transition across (if ever).

            Rules are important, but given this was a once-in-a-century level of global banking crisis, I think a bit of pragmatism would have served better. Worrying about moral hazard was what caused the US to let Lehman Brothers collapse and the Bank of England stuff Northern Rock. Nothing good came of those 2 decisions. Similarly the rigidiy in stopping Italy from trying to solve its banking crisis has stored up potentially horrible trouble for later.

            Presumably the main blame still goes to TSB and Sabadell management for doing the migration on the cheap or too fast.

          3. Anonymous Coward
            Anonymous Coward

            Re: What are IBM going to do? Wave a fucking magic wand?

            > IBM are most likely the best people to fix it because it's IBM technology that's involved.

            Just because IBM made something doesn't mean they're competent with it past the sales point.

            Remember the many waves of outsourcing support, and deliberately removing their more experienced technical staff? (still ongoing)

            Bringing in IBM is unlikely to achieve an meaningful win, apart from giving the CEO someone else to blame.

          4. Vincent Ballard

            Re: What are IBM going to do? Wave a fucking magic wand?

            My inside information from a former colleague is that Sabadell tried to port their existing code for their Spanish bank, which was cowboy quality. Hard-coded values including IP addresses for servers and telephone area code prefixes. Copyright-violating ripoff of a Netflix library from GitHub where the copyright header was changed but not the references to Netflix in error messages. Compounded by the insistence of the suits that 500 simultaneous users was a sufficient load test.

            1. Anonymous Coward
              Anonymous Coward

              Re: What are IBM going to do? Wave a fucking magic wand?

              Please do stop.

              ROFLMFAO

            2. Anonymous Coward
              Anonymous Coward

              Re: What are IBM going to do? Wave a fucking magic wand?

              "port their existing code for their Spanish bank"

              That would seem to be confirmed by the BBC report of one TSB branch manager's bad week - they said that the branch systems have been spitting out Spanish-language error messages when things inevitably fail or time out

    2. Dan 55 Silver badge

      Re: What are IBM going to do? Wave a fucking magic wand?

      No, IBM are there to save Pester's CV when he inevitably gets the boot.

      IBM save the day? "I took that decision. Me! Me! Me!"

      IBM don't save the day? "Not even IBM could fix that screw-up of a system that Sabadell made me use."

    3. Anonymous Coward
      Anonymous Coward

      Re: What are IBM going to do? Wave a fucking magic wand?

      No, the CEO has lost faith in IT who say they will fix it ASAP. He now realises they dont know what they are doing so is now desperate so bringing in someone who can give him the truth.

    4. d3vy

      Re: What are IBM going to do? Wave a fucking magic wand?

      " just means that the people who are supposed to be fixing the problems spend pointless time explaining things to the newbies."

      That in itself might be a useful excercise. It's even got a name. Rubber duck debugging.

      Sounds stupid but it works.

  3. Anonymous Coward
    Anonymous Coward

    I wonder...

    This does sound a lot like that ding dong at that Belgian bank a couple of weeks ago.

    1. I ain't Spartacus Gold badge
      Happy

      Re: I wonder...

      Was that the one where Leslie Phillips tried to seduce the manager's wife in order to get the key to the vault?

      Oh, I say! Ding... Dong...

  4. Steve Graham
    IT Angle

    I was once manager of a team which was migrating more than 25 million customers to a new infrastructure overnight. I was so confident that we had covered all contingencies, including total roll-back, that I went home and went to bed.

    It went perfectly.

    It wasn't easy, and it wasn't cheap, but failure is always avoidable.

    1. Chris Tierney

      Excellent attitude. Demonstrates absolute trust in your team and for that they'll likely work harder for you to make sure it all works.

      1. WallMeerkat

        Seriously?

        If I was up all night on a stressful cut over migration of 25 million accounts while my manager skipped off home to bed it would be anything BUT an "excellent attitude".

        There is trust in the team (do not micromanage) but it's nice to know the general is beside you when the troops are at the front line.

        1. Doctor Syntax Silver badge

          "it's nice to know the general is beside you when the troops are at the front line."

          Even if he's only refilling the kettle and cutting the sandwiches. In fact especially if that's all he's doing.

  5. Doctor Syntax Silver badge

    Oh, the irony

    Wasn't the whole reason for splitting TSB from Lloyds to avoid the situation of having banks that were too big to fail?

    1. katrinab Silver badge

      Re: Oh, the irony

      TSB is small enough to fail.

  6. tip pc Silver badge
    Facepalm

    1 way migration

    Looks like this was a last minute 1 way migration with extremely poor testing and over site.

    T minus 7 days: heads we do the change tails we don’t.

    T minus 5 days: any plans this weekend, weathers meant to be good. what about this change? The change that’s always delayed? Apparently the ceo is adamant this time!! I’ll beleive it when I see it.

    T minus 4 days: notify the customers it’s happening, cancel your weekend plans!!

    T minus 2 days: we’re down 10 senior tech lead heads for this weekend, 1’s getting married, another 2 on maternity, 5 on leave and the other 2 are “busy”. Everyone is saying they are not ready but the pm’s think the remainder can get the job done.

    The rest is history.

    I couldn’t imagine doing something like this in 1 go over 1 weekend without a credible blackout plan.

    You’d pause all internal transactions and record all externals, do your migration, test the hell out of it, then roll it back and replay transactions And unpause stuff, then during the following week analyse the tests to check for problems, repeat until all the kinks are ironed out then roll out to a small number of customers per weekend, looking and checking for issues including fraud.

    All a very simplistic high level glossing over huge amounts of detail but my fag packet looks a lot more detailed than their packet of Rizla with roaches torn off.

  7. Anonymous Coward
    Anonymous Coward

    I didn't trust Lloyds after they screwed over a family member

    I don't trust Lloyds TSB and I see no reason to trust TSB

  8. HmmmYes

    This seems to be evolving - or going to shit.

    http://www.bbc.co.uk/news/business-43904267

    'TSB chaos: 'We are on our knees,' says boss'

    A few days ago they are getting their bonuses for a completing the work.

    Yesterday it was just a couple of niggles affecting a handful.

    Today, Godzilla is running rampant thru the server room.

    1. Julian 8 Silver badge

      Now IBM have joined in, it has gone pete tong

    2. Anonymous Coward
      Anonymous Coward

      I was thinking that what's s the betting some of the trouble is being a Spanish bank bits of the software will be in Spanish, adding to the levels of bullshit. then I read this

      http://www.bbc.co.uk/news/business-43892840#

      "He reported that branch staff were trying to cope with a completely new system after the botched migration but "nothing is working".

      "My concern is we have customer bills to pay such as credit cards, but we physically cannot process them, as the system either freezes or completely crashes, like it did yesterday for the whole afternoon."

      He said that staff had been assured for more than a year how amazing the system would be, but "it's an absolute joke. Most of the error messages are in Spanish, which is great if you can speak Spanish!"

      1. steviebuk Silver badge

        Also concerning is "Staff were trying to cope with a completely new system". TSB have never heard of training then.

        1. This post has been deleted by its author

        2. Adam 52 Silver badge

          There's a difference between being able to operate a new system day-to-day, which you may have been trained for, and being able to work around a system you've been using for years and know all the back routes in.

          We have a mission critical but old, command line system at work. It's massively reliable (a few minutes downtime in three decades), fast and really efficient for those trained in it but management can't cope without drop-down menus so is being replaced with a much less reliable, much less capable Windows app. Thank's for nothing Steria.

      2. Anonymous Coward
        Anonymous Coward

        Es asombroso y roto!

  9. HmmmYes

    I like this:

    'Mr Pester, said he would take direct control of the issue, and had drafted in experts from IBM, who would report "directly" to him.'

    Err, Mr Pester, you are the at the root of the problem.

  10. Anonymous Coward
    IT Angle

    Cash cow!

    I hope the time-served IT staff who were made redundant in favor of 'Steve' in Mumbai won't lift a finger to help for less than £1,000/day

    1. Dan 55 Silver badge

      Re: Cash cow!

      They can name their price but the trap is they'd have to get it paid into a TSB account.

    2. Anonymous Coward
      Anonymous Coward

      Re: Cash cow!

      I'm waiting for that call.

  11. adam payne

    But the bank claimed that "each of these instances were 'connected accounts' where third party access had been granted, typically to family members" and that no personal customer information was shared.

    Bold statement but are you really sure? Wouldn't want that to come back to bite you are anything.

    They hired IBM! Oh dear, they really do have no idea how to fix it.

    1. yoganmahew

      @Adam

      "They hired IBM! Oh dear, they really do have no idea how to fix it."

      There can surely be no greater admission of failure, can there?

  12. HmmmYes

    And this is something that was planned and executed entirely within the bank.

    How will they ever deal with a cyber attack.

    1. I ain't Spartacus Gold badge
      Devil

      How will they ever deal with a cyber attack.

      What are you talking about? They're the only bank in the world that's totall safe from any cyber-attacks!

      This is just the logical extension of the saying, "my network would be perfect if only it didn't have any users."

  13. wolfetone Silver badge

    I think I must be the only customer TSB has that's finding this all rather funny.

    You'll find me in the room that's covered in fire, just chanting "THIS IS FINE".

    1. Jason 24

      I am too, but then they aren't my primary bank, just use them to hold a couple of grand for the 3% interest.

      Very happy to see it going back to 5% after this!

      1. Anonymous Coward
        Anonymous Coward

        I was a customer for a couple of years. Coincidentally my switch date was last Friday so just missed the fun and games.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like