HSBC e-payments system goes titsup (again) Merchants are furious at HSBC UK after the bank's e-payments system fell over for an extended period for the second time in less than a fortnight. Retailers using the system have not been able to process transactions since around 5pm (BST) on Monday. Problems with processing credit card payments continued into Tuesday …
while its not acceptable for HSBC to be down like this and not keep clients updated, there is also a finger to be pointed at clients who rely on this exclusively and dont have any alternative.
If something is _SO_ crucial to your business, why no backup option?
I can never understand this.
Anything to do with computers can and will go wrong eventually. Fact of life.
If you could potentially lose thousands at any outage, then its your responsibility to have backup plans. No excuses. Even if its just paypal. anything.
Paris knows alot about titsup, i'll bet
Banks may have fancy web sites, but their IT systems are still ancient.
Their IT departments are run like civil service departments - lots of meetings , lots of changing priorities, and very little done.
The real work gets done by external contractors and a few skilful souls in the bank who have to battle against blind incompetence and 'job for life' managers who prefer trying their luck with the call-centre staff, the silly old fools..
HSBC e-payments system is horrific to integrate with, to add another service and be able to switch between the 2 adds yet another level of complexity. Sticking a paypal button might seem like a good idea but
a) you'd lose cusotmer confidence more than just saying "we're having a problem"
b) you can't do transactions such as pre-Auth and Post-Auth, which we use so that the card is charged correctly post-delivery.
Currently, we're developing a buffering system that hopefully(!) will allow us to buffer transactions bound for pre-auth when HSBC invariably go down again.
Although, by that time, we're likely to have moved.
www.wiltshirefarmfoods.com
I have to agree with Seanie that people should have some form of business continuity plan should a service or services like these fail. Even taking manual payments with a PDQ keeps ya going!
It would not be hard to integrate a second gateway into most if not all shops/stores/whatever and fallover to that should or when HSBC breaks. You get paid into the same account it is not a major issue then.
Look at the mess with Protx a while back when people were screaming the same thing that they were loosing £10,000s as it was offline - surely if your putting that much into a gateway you 'should' have a backup.
And surely as a "business" you don't have one copy of your accounts ... payroll ... customers list ... Do you????
Mind you lost faith with integrating HSBCs payment thing ages ago. Far faster ones exsist nowadays thank god.
I couldn't agree more.
It's called business continuity planning, and most companies never bother to think about it or realise that it's something, just like all the other costs of doing business, you need to pay for. The problem is that doing business via the web has been tarted up as some kind of panacea of fault tolerancy and commercial utopia, when in fact it's exactly not. Businesses are brainwashed into thinking that they should pay little or nothing for a 24x7x365 presence to run their web shop fronts.
It never fails to amaze me the irate calls we get claiming losses of 10's of thousands of pounds of sales if one of our shared (tenner a month) boxes hits some problem for a few minutes. Yet when we offer them a dedicated box with failover which probably costs them less a month than the leasing payments on their Jags and Astons sitting in their company parking spaces they suddenly lose interest.
If your business is that bloody important and you do it on the web make sure you have two of everything (at least) - and that includes an alternate payment gateway (which may cost you a bit more per transaction, but at least you can still take a payment/authorise in real time).
Paris, because I bet she's got two of everything.
"The real work gets done by external contractors and a few skilful souls in the bank who have to battle against blind incompetence and 'job for life' managers who prefer trying their luck with the call-centre staff, the silly old fools.."
Is this supposed to be tongue in cheek?! Has this idiot ever worked for a Bank? I love the attitude of people who feel this could never happen to them. Has it ever occurred to anyone that it could be an external vendor at fault?! It's all too easy to criticise without the necessary facts to hand...
-->It never fails to amaze me the irate calls we get claiming losses of 10's of thousands of pounds of sales if one of our shared (tenner a month) boxes hits some problem for a few minutes.
but when your annual HSBC processing bill comes to just under £20k - you can see why you could get a bit irate
integrating with another system to provide failover would cause more problems that it solves.
For example with HSBC you get a response code - 22 I think it is, to say that the customer's supplied postcode doesn't match their card accounts. If you then integrate with RBOS or whoever, who return code NoMATCHPOST instead - you have to have logic for every type of transaction and every possible outcome, twice, all the time relating it to which processor you're using at the time, just becuase the major international bank you've been using can't get it's arse in gear.
If your payments system is suitably abstracted, handling different responses from different gateways should be a simple case a of writing a translation layer.
The additional time and cost spent implementing a second payment gateway far outweighs the risk of not being able to process transactions.
If your business depends heavily on taking online payments you'd be crazy to rely on a single provider, no matter how good their track record. It's the same rule that applies to hosting, backup and pretty much anything else that can fail.
It's easy to close an HSBC account; you go into a branch, and say "I'd like all my money please. Bankers Draft. If you can't do that because of a computer error, cash will be fine. I assume you can still count without using a calculator?"
You then promptly take your money down to another bank, and deposit in a new account. Setting up the services you need may take longer, but there are alternatives in the mean time. Assume HSBC have outsourced the fault finding to India, and they're using a remote desktop session over a 28.8k modem, and a local engineer whos qualifications top out at "have upgraded the ROM in a CPU."
The important thing is your money is not with HSBC, and that's what they don't want to happen.
We have had a backup system in place on our API based HSBC system that checks to see if the HSBC server is there using CURL. If you are a developer then look into this. You can then allow your system to automatically decide if the payment is taken there and then or it stores it for later, so that it can be manually processed when the system comes back up.
Yes we have lost thousands in the past from HSBC but sh1t happens people and it will continue to happen all over the world with many things. Take some time off tomorrow and plan your contingency, do not waste time on complaining!
I have been in contact with a few "insiders" who have been able to confirm the cause of the problem.
After the recent loss of CDs they decided to upgrade their systems to use DVDs, they used up their stocks of CDRs and were due to go live with their new improved DVD based communications system for handling transactions - however they realised a bit late that the server only had a CD burner, not a DVD burner and naturally they had no spare CDRs to revert to the old system.
The delay in getting back online isn't being helped by the fact that for some reason their supplier is not taking any orders due to some sort of payment processing outage.
You still dont get it do you?
Lets say that next week NoChex decide to do an upgrade and it all goes wrong.
What then? It _could_ happen.
You have just seen how easy it was to switch. Why didnt everyone who is complaining setup a nochex account months ago and have, at the very LEAST, 2 sets of html/php/asp pages, so you can put your nochex in when you want, and your hsbc (or other) when you want. Its that easy guys!!.
I just hope you all kept your HSBC account active and a copy of the integration.
That means you NOW have a backup for the future. Pat yourself on the back.
hope NoChex doesnt buckle under the new load !! ;-p
Hong-kong/Shanghai Banking Concern is not the combination of words I first think of when the acronym HSBC is mentioned.
I shall leave it to the collective imaginations of the assembled to come up with more...accurate...interpretations of their initials
They have mismanaged just about *every* aspect of my company's account since it's inception in December 2005.
So this comes as no surprise whatsoever.
Cunch of useless bucking funts
If you're doing 10k of business a day, a 20k bill from HSBC is a drop in the ocean....and it's called 'cost of sale' which so many retailers fail to understand....but that's another rant for another time.
In answer to "integrating with another system to provide failover would cause more problems that it solves.", as Whelk correctly states, you wrap your payment processes up in an API that abstracts away the specifics of the payment service.
Yes it was slightly tongue in cheek, no I don't know the specific facts of the case, and yes I have worked for a bank - for a very short period of time (NR as it happens - ho ho ho).
But you are quite right - I should know better. On the other hand, I think there was more than a grain of truth in what I said. Apologies to all hard working sould who work in banking IT depts. Blame the managers is always my watchword - they get paid to do the planning after all...
They've just been hit by one of the biggest losses in the sub-prime crisis. Those poor things are still reeling from the loss of their massive bonuses and their annual junket^H^H^H^H^H^H board meeting in their Bahamas Headquarters !!
Please take pity on those poor, deprived persons struggling to make do with their last Rolls Royce and multi-million quid country home !!
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
