NatWest customers struggled to access the company's online banking, ATMs, telephone and even branch systems in the past few hours, after it was hit by a unspecified "technical issue" this morning. A spokeswoman at the firm denied that the bank's systems were hit by an outage, and insisted to The Register that NatWest had simply …
EGG also having problems
Neither me nor my wife can access our Egg accounts (well, when you log in and try and click on an account, it gives an error message telling you its unavailable). Checking twitter there are several messages (search "Egg Credit Card") telling similar stories so I don't think its just us.
Are all Banking computers taking a Friday off perhaps?
I've also just noticed that RBS are also having similar issues, I have accounts at both Natwest and RBS, and both are so slow to log in I can't without it timing out. They both use the same software (judging by the near identical interfaces), I presume they are using the same backend too.
RBS System off as well
System is running really slow - 10 minutes or more per page load. Payroll for many is totally ****** up today.
Has been this way since at least last night although they just seemed to start to notice around midday today.
Don't even get me started about the moronic web interface itself..........
I assume that even though there are very few of them left, and their systems appear to be dying, that all the hard working IT staff will be receiving huge bonuses?
I withdrew cash from my NatWest account not 1 hour ago.
Maybe it's fixed, maybe it was never broken, either way I don't care :D
No problems my arse
NatWest online banking is unusably slow right now
They're making progress..
I can now put in my customer number.. Maybe by next Tuesday I can view my balance..
Bill G, cos his software must have something to do with this..
Not working for me either
I have been having problems too. First tried to access it about 10.30am natwest.com was loading but the online banking side wasn't. Been trying throughout the morning and still it not loading.
Only just got online banking with Natwest
Got my first 'check your statement' email today, went to login, timed out myltiple times.
I tried http://downforeveryoneorjustme.com/ and it said the site was up. I assumed that they just had a shit system and sent all their customers the email at the same time and everyone was trying to check their account.
Its good to know it wasn't just my connection.
I've been using it for several years
and normally it's rock-solid, hence the consternation at today's outage.
"NatWest suffers calamitous online banking breakdown"
Not the slightest vestige of exaggeration there?
People can't access their bank account for an hour or two?
The end of the world is nigh. Reboot the universe...
Well, it's been down all day
and I've got bills and rent to pay. While not the end of the world, I'll certainly regard it as annoying when I start getting eviction notices.
Worked for me this morning, also at 12:30.
I think they love me.
A slow response time across a decent time period still counts as a service outage in my books.
Of course there's a distinction if you're trying to diagnose the issue, but if it's crazy slow then it's still broken from a user's perspective.
I'd say it's borderline at the moment - certainly slow, but usable.
MBNA as well
MBNA have just told me I can't log in as their systems are down for maintenance - on PAYDAY ?!?!
iPhone app failing
Can't get intoy account with the iPhone app... The server gives an invalid response
It doesent work like that?
Wasnt that their advert line??
Wasn't working at 10:40AM and still not working now
I dunno how people can access the site at all -- I guess they are just lucky! For me, the online banking site won't even load now: https://www.nwolb.com/
I did manage to get to the second page of login earlier today, but it hung after I'd entered the various letters of passphrase, and I daren't try again or it might log multiple failed login attempts or something.
Also, why the f** don't they have a service status page?
RBS Redundancy day celebrations..!?
Hmm. Maybe a coincidence....but...today is the day a lot of RBS "Business Services" staff leave the Bank. These are the people who do techie stuff and the like.
Maybe they are all just moving their bank accounts elsewhere ?
So when exactly do you fail over?
Being a financial institution (so I'm told) they have to have a disaster recovery plan (again: hearsay). So they can provide a service after .... well .... a disaster. The question then becomes what exactly is a disaster and when do you invoke the procedures?
For an organisation that relies on 'pooters as much as Natwest or any other bank does, you'd kinda hope that this process would be fast, slick, reliable, well rehearsed and most of all unnoticed by the punters. The two questions that need to be answered therefore, would be: when do you say "okaaay, we're up the creek - someone press the big red button" and also "has anyone ever tested this before"> Since DR processes are like backups: until you try them, you have no ideas if they work.
If I was in charge of Natwest's IT systems, I'd use this as an opportunity to give it a try - with a nice fat bonus for the DR manager in one hand, and his/her/its letter of resignation in the other.
Any financial institution has to fail key systems over to DR once a year to prove it works (I believe this is a regulatory requirement) So there would be no need to "give it a try". Also, when recovering a complex distributed system like an online banking application, you don't just randomly move bits to operate in different datacentres on the offchance that you make the situation worse.
The recovery management team will be looking into what is causing the problem, it may well be that moving to a disaster recovery site wouldn't resolve the problem. Possibly it's a problem with application code, possibly it's a problem outside of the datacentres with the internet itself. It may be a firmware bug on some backend system which is running identical hardware/firmware to the DR site, so failover wouldn't help. Hell, they may even be under a DDOS. Failover to DR isn't always the answer.
Re: What is a disaster?
Disasters come in all sorts of shapes and sizes... everything from terrorist attacks to the armored truck carrying the chairman's bonus breaking down :-)
After 9/11 I know of several colleagues who camped out and slept on the machine room floor on Wall St for 48 hours keeping transactions going, because they knew if they left they wouldn't be able to get back in.
But yes, they do test DR processes, and if a system in London goes down it's quite likely a support guy (or gal) in Singapore or Chicago is the one who ends up fixing it.
Live / Live
Please go away and learn how to set up proper IT systems
I'm always amazed at the "IT Specialists" who make stupid comments on here about DR plans, backups, AV , etc. There are BIG organisations out there who know FAR more about running these services than your piddly little IT group that supports 20 servers and 300 users.
I think you'll find most of the really critical services run live/live out of multiple locations, so a DR plan typically doesn't have a big red invoke button, but simply a "let's recover that bit that's failed in a controlled manner while the rest of the system picks up the load" - and yes, that plan is written, and tested twice a year as part of their license from the FSA.
Enterprise IT is pretty good at making sure the infrastructrure works, and since banks have been amongst the biggest users of IT for nearly the longest time, they generally have all the bases covered (yes, yes, occasionally some Business idiot manages to squeeze in some "service" that doesn't work properly)
Unfortunately the likes of IBM and HP got to the banks first. Their idea of disaster recovery is to fit a massive, expensive mainframe with fault tolerant hardware and have a hourly (or even daily) backup of the data.
The managers can argue that the hardware can never fail so there is no point to having a fail over site constantly ready. Should the primary system actually fail then it can take hours if not even days to get the DR site on line.
Of course, software bugs are every where and fault tolerant hardware is not (HP Non Stop sometimes does stop), so every so often this happens, but you would be surprised how resilient these systems are. Think 10 years up time...
Personally, I write software tolerant banking systems which can switch loads between data centers in less than 30 seconds. Take that Google!
On a tangent, I once saw a multi-million dollar Tandom go down because some penny pinching manager hadn't bothered to replace the $5 backup fan. When the primary failed due to old age the system went down for a day and he lost his job. Shame.
Oh no they don't...
At a finance instituiton I worked at, they DR test the back end but never the customer facing bits. Probably in case it doesn't work and causes an outage. And no, this is not a penny ante organisation. It's one of the biggest in the UK (well into the many thousands of millions)
Luckily the Reg has never noticed when the online service goes down, don't ask me why, they just never spot it.
Some oddities still
We bank at Coutts, which I think uses the same backend.
In the last hour, my partner failed to authenticate on multiple occasions. And got herself locked out. I had to restore her access. Twice.
Suddenly she could login.
Then *I* couldn't damn well login.
Then, suddenly, I can again.
Something, somewhere is still very screwy.
...takes botnet off line. Just a coincidence?
Read the article
and all the comments and the login page still hasn't loaded.
However Mobilink is working OK
The non-payment transactions
Huge backlog of BACS/CHAPS payments leaving Natwest accounts as they have had to freeze the process. Monies are being debited but not ending up in destination accounts.
So where is the money going?
What sort of idiot...
...doesn't have at least a day's cash in their wallet in case their bank's pooters fail ?
Oh yeah, I live in the UK. I'm surrounded by early-adopters who expect every bit of technology to work all the time, and are also paranoid they'll get mugged if they have more than 50p on them.
Paris, cos when she goes down, it'll last 10 minutes at most.
It's an ingenious idea to claim it's not unresponsive, just a relay delay --- students could use this when finishing time looms.
I know the reason for this.........
RBS had to pay Fred Goodwin his Pension this month so maybe they have run out of OUR money ?
We need more of these, actually
Simply because we're actively being pushed into ``going electronic'' to the exclusion of cash (Dutch National Bank, I'm looking at you here), with no options left when, not if, something happens.
This something may be outages at the bank, or it might be someone used the automatic debit system to relieve you of your cash, or some faceless clerk caused your account to be blocked due to suspected abuse or a possibly misfiled order to judicially collect monies or I don't know what else might go wrong. I do know the effect: No access to your money to pay for food or rent, and an expected ETA of restored service that may easily be several months in the future.
RBS vs Natwest
Duh - its the *same system* behind the scenes. The two internet banking sites are identical apart from branding imagery.
That explains the fail.
Well this explains why I couldn't get in to my Ulster Bank Anytime Banking online, but thankfully I didn't have the other associated issues with the NatWest/RBS mega fail.
At least when I called in I could get my balance to piss away my not-so-hard earned cash.
No, no! 'E's resting!
ah...probably pining for the fjords.
reply to Steve Evans
Steve, im sure your post was supposed to be comical but believe me us in the IT areas of RBS DONT get big bonus' at all.. in fact in the 12yrs i have worked for RBS ive had one bonus which i think was way back in 1998 and was about 200quid. Most of us "proper workers" are just pleased we still have jobs. How the company is run is like anywhere else so dont tarnish us all with the same brush as the people at the top making all the decisions!!
The Windows Updates are finished
Looks like everything's back to normal again, I logged in to my NatWest account first time just now (and the web site seemed pretty snappy compared to its previous performance). Perhaps they're running Dual Cores now?