Re: not a customer
I was a customer of TSB for many years/decades.
Then after the merger I became a customer of LloydsTSB.
Then when TSB was sold off I emerged as an involuntary customer of Lloyds.
Dodged a bullet.
In the midst of its six-day online banking meltdown, which is showing no sign of letting up, TSB has hauled in the systems integration big gun or as it is otherwise known, IBM. The British bank confirmed the signing of Big Blue this morning as part of its rather uncomfortably timed first quarter financial results for 2018, in …
"I'm a TSB customer and as far as I can see, my direct debits and standing orders have worked just fine since Monday morning."
They have been "Core services" were up and running at the weekend... but no one can access their accounts.
I have a few accounts with TSB one which I use day to day and one for the bills. At the end of every month I work out the next months outgoings and transfer them into the bill account which I then dont touch - this means that no matter what happens during the month the bills are all covered.
The problem now is that as its the end of the month that bill account is empty, I cant put any money in it because the account with the money in is a TSB business account (and the internet banking for the business accounts is in a worse state than the normal accounts).
So shortly my mortgage and a host of other payments will come out of my bill account - this will do one of two things (Maybe both)
1. Put me WELL over my overdraft
2. DD will get declined and go unpaid.
"sounds like a complex undertaking done for good reasons."
Well, You're right to an extent.
It IS a complex problem and they do need to do it.
"Customers should just be patient and not get agitated."
I wasn't agitated until at 7pm on Sunday (AFTER the switch over was meant to be complete) when my debit card stopped working (Its still not working now).
ATMs also seem sporadic - Though to be fair I've only tried once.
I can tell you now that 6 days on I am more than agitated I have two personal accounts one which is used just for bills and by this time of the month will be empty. The other has money in it but I cant get to it.
I also have three business accounts which I cant get to to pay wages out.. or check that Ive been paid by clients..
So my only option now is to go into a branch and HOPE that they can transfer the funds for me - from what I have read online this is a bit hit or miss at the moment.
Now not only do they have a badly planned, poorly managed cut over to repair they are also going to have a bunch of senior muppets from IBM looking over their shoulder second guessing them with the benefit of hindsight.
I bet somewhere in the depths of TSB's tech dept there are a number of people muttering "told ya so" under their breath who were ignored when they raised issues.
I don't see what value IBM can add at this point other than making things worse. I get a very strong whiff of "something must be done"
Anonymous Coward is spot on. A former colleague of mine is caught up in the middle of this, having joined Sabadell recently in a QA role, and reports that people are pulling seriously illegal amounts of overtime and staying awake with the aid of pills and Bolivian marching powder.
"Even if the problem was caused by management ignoring advice from techies, it'll still be the techies fault ... somehow."
Space Shuttle. O-rings.
You'd think those who don't know how things work would have seen that one as a Really Big Wake-Up and would now be paying attention to those who do Know Things.
Or am I expecting too much from Admin and Management Plonks?
So how do they expect a bunch of outsiders to come along and fix everything when their own internal people who built the fucking thing can't get it working quickly?
Bringing in outsiders always looks good to the mouth-breathing morons up the org chart, but doesn't fix the problem just means that the people who are supposed to be fixing the problems spend pointless time explaining things to the newbies.
This Peter Pester twat sounds like a complete waste of space.
There are a lot of people in this thread who are jumping to some remarkable conclusions.
A couple of disclaimers - I have no inside info here, and I once worked for IBM (left a few years back).
Everyone here's pointing the finger at TSB and their new owners here but the reality is most likely that this is all a great example of the evils of lock-in, which can be entirely attributed to Lloyds. There is a lot of IBM presence in that bank and they are very dependent on overpaid contractors to keep it running, along with oodles of offshore bollocks.
My guess (and it is admittedly a guess, although I hope an educated one), is that TSB are trying to change as little as possible, so are splitting out IBM systems, or at least duplicating them, and someone somewhere has not done the numbers, or more likely (in my vast experience) has been told those numbers (performance metrics, technical necessity etc.) do not match the other numbers (£££) and some serious corners have been cut. I hope the relevant technical people kept their emails on the subject.
IBM being brought in now is unlikely to be a free market choice; IBM are most likely the best people to fix it because it's IBM technology that's involved. If it's an architectural issue or a sizing issue, as is most likely, and this was not IBM's fault, then IBM aren't going to fix it without getting paid. Support doesn't cover that. The fact that TSB have little choice other than calling IBM is entirely down to lock-in. Of course, they may have had the option to move off IBM here, but probably decided to change as few things as possible to improve chances of success.
I'll reiterate that this is conjecture on my part, but I have a lot of experience here - being sent halfway round the world to fix customer issues. Of course, I no longer work there (took a package) but there are still people who can probably fix this. It won't be quick though and I can guarantee that the people who do fix it will get zero credit. Which is one of many reasons I no longer work there.
I am hoping the details do get leaked, but I for one am not pinning this entirely on TSB.
"I am hoping the details do get leaked, but I for one am not pinning this entirely on TSB."
What has to be pinned on TSB is that they took stock of the situation and decided they were in a position to go ahead in a single operation. And then found they weren't.
It's a bit of an EU cock-up in my book too. RBS have apparently wasted spent a couple of billion trying to sell several hundred branches to comply with competition rules - but nobody in the end wanted to buy them. So they're now spending billions more trying to spin them off as Williams and Glyn. That means setting up a head office, management, IT, and a cash pile/reserves.
This TSB thing is a similar exercise in regulatory box-ticking. That's doing nobody much good. How much have we really gained as customers attracting another Spanish bank into the retail market, as a minor player with just a few hundred branches?
We had state aid and shotgun mergers during the financial crisis and that broke competition rules. But they really should have ignored them. At least until a later date. There's a reason that when banks buy other banks they tend to leave them on the old systems for years, and only slowly transition across (if ever).
Rules are important, but given this was a once-in-a-century level of global banking crisis, I think a bit of pragmatism would have served better. Worrying about moral hazard was what caused the US to let Lehman Brothers collapse and the Bank of England stuff Northern Rock. Nothing good came of those 2 decisions. Similarly the rigidiy in stopping Italy from trying to solve its banking crisis has stored up potentially horrible trouble for later.
Presumably the main blame still goes to TSB and Sabadell management for doing the migration on the cheap or too fast.
> IBM are most likely the best people to fix it because it's IBM technology that's involved.
Just because IBM made something doesn't mean they're competent with it past the sales point.
Remember the many waves of outsourcing support, and deliberately removing their more experienced technical staff? (still ongoing)
Bringing in IBM is unlikely to achieve an meaningful win, apart from giving the CEO someone else to blame.
My inside information from a former colleague is that Sabadell tried to port their existing code for their Spanish bank, which was cowboy quality. Hard-coded values including IP addresses for servers and telephone area code prefixes. Copyright-violating ripoff of a Netflix library from GitHub where the copyright header was changed but not the references to Netflix in error messages. Compounded by the insistence of the suits that 500 simultaneous users was a sufficient load test.
"port their existing code for their Spanish bank"
That would seem to be confirmed by the BBC report of one TSB branch manager's bad week - they said that the branch systems have been spitting out Spanish-language error messages when things inevitably fail or time out
" just means that the people who are supposed to be fixing the problems spend pointless time explaining things to the newbies."
That in itself might be a useful excercise. It's even got a name. Rubber duck debugging.
Sounds stupid but it works.
I was once manager of a team which was migrating more than 25 million customers to a new infrastructure overnight. I was so confident that we had covered all contingencies, including total roll-back, that I went home and went to bed.
It went perfectly.
It wasn't easy, and it wasn't cheap, but failure is always avoidable.
Seriously?
If I was up all night on a stressful cut over migration of 25 million accounts while my manager skipped off home to bed it would be anything BUT an "excellent attitude".
There is trust in the team (do not micromanage) but it's nice to know the general is beside you when the troops are at the front line.
Looks like this was a last minute 1 way migration with extremely poor testing and over site.
T minus 7 days: heads we do the change tails we don’t.
T minus 5 days: any plans this weekend, weathers meant to be good. what about this change? The change that’s always delayed? Apparently the ceo is adamant this time!! I’ll beleive it when I see it.
T minus 4 days: notify the customers it’s happening, cancel your weekend plans!!
T minus 2 days: we’re down 10 senior tech lead heads for this weekend, 1’s getting married, another 2 on maternity, 5 on leave and the other 2 are “busy”. Everyone is saying they are not ready but the pm’s think the remainder can get the job done.
The rest is history.
I couldn’t imagine doing something like this in 1 go over 1 weekend without a credible blackout plan.
You’d pause all internal transactions and record all externals, do your migration, test the hell out of it, then roll it back and replay transactions And unpause stuff, then during the following week analyse the tests to check for problems, repeat until all the kinks are ironed out then roll out to a small number of customers per weekend, looking and checking for issues including fraud.
All a very simplistic high level glossing over huge amounts of detail but my fag packet looks a lot more detailed than their packet of Rizla with roaches torn off.
This seems to be evolving - or going to shit.
http://www.bbc.co.uk/news/business-43904267
'TSB chaos: 'We are on our knees,' says boss'
A few days ago they are getting their bonuses for a completing the work.
Yesterday it was just a couple of niggles affecting a handful.
Today, Godzilla is running rampant thru the server room.
I was thinking that what's s the betting some of the trouble is being a Spanish bank bits of the software will be in Spanish, adding to the levels of bullshit. then I read this
http://www.bbc.co.uk/news/business-43892840#
"He reported that branch staff were trying to cope with a completely new system after the botched migration but "nothing is working".
"My concern is we have customer bills to pay such as credit cards, but we physically cannot process them, as the system either freezes or completely crashes, like it did yesterday for the whole afternoon."
He said that staff had been assured for more than a year how amazing the system would be, but "it's an absolute joke. Most of the error messages are in Spanish, which is great if you can speak Spanish!"
This post has been deleted by its author
There's a difference between being able to operate a new system day-to-day, which you may have been trained for, and being able to work around a system you've been using for years and know all the back routes in.
We have a mission critical but old, command line system at work. It's massively reliable (a few minutes downtime in three decades), fast and really efficient for those trained in it but management can't cope without drop-down menus so is being replaced with a much less reliable, much less capable Windows app. Thank's for nothing Steria.
But the bank claimed that "each of these instances were 'connected accounts' where third party access had been granted, typically to family members" and that no personal customer information was shared.
Bold statement but are you really sure? Wouldn't want that to come back to bite you are anything.
They hired IBM! Oh dear, they really do have no idea how to fix it.