back to article Day 2: Millions of HSBC customers still locked out of online banking

Potentially millions of customers are still unable to access their HSBC accounts online, with the service outage now in its second day. As The Register reported yesterday, the problems occurred at 8am Monday (4 January). The bank has ruled out a DDoS attack as the cause, but has not elaborated further. Business Banking …

  1. Timbo

    still not working, even for BIB..

    "Business Banking services are currently running at a significantly reduced capacity and running slower than normal, said the bank in a statement this morning."

    All I get when I try BIB is:

    "This web page is not available"

    followed by:

    "The server at www.business.hsbc.co.uk can't be found because the DNS look-up failed."

    oh dear !!

    1. AMBxx Silver badge

      Re: still not working, even for BIB..

      Worked for me at 8am, not since though. Doesn't sound like a DNS problem, but never know.

      Just annoyed that it gives customers yet another excuse for late payment.

    2. Vic

      Re: still not working, even for BIB..

      "The server at www.business.hsbc.co.uk can't be found because the DNS look-up failed."

      It looks like they've been having fun with DNS :-

      www.business.hsbc.co.uk. 20 IN A 193.108.75.150

      20s TTL for the A record? That's nasty...

      Vic.

      1. david_n_m_bond

        Re: still not working, even for BIB..

        Wow - down to 13 seconds now. Are they using DNS for load balancing?

      2. david_n_m_bond

        Re: still not working, even for BIB..

        Vic,

        How are you getting 20s? If I execute:

        nslookup -type=a -debug www.business.hsbc.co.uk

        ...at a Windows command prompt, I'm getting between 0 and 20 seconds, seemingly randomly, though it's always returning 193.108.75.150 for me. Also, can you think of ANY scenario where this might be appropriate? Perhaps a load balancing scenario? Otherwise, could it be a DNS traffic redirection attack?

        David

        1. Vic

          Re: still not working, even for BIB..

          How are you getting 20s?

          I'm executing "dig +trace www.business.hsbc.co.uk".

          I'm getting between 0 and 20 seconds

          You're not doing a full recursive lookup; you're just querrying your upstream servers. So the first time you try, there is no record there. and that server performs the lookup. For the next 20 seconds, you get the remnants of that record (which is why the TTL drops below 20s), until is is expired. At that point, your enxt request starts the lookup cycle again.

          Use the "recurse" switch to avoid this behaviour.

          Also, can you think of ANY scenario where this might be appropriate?

          Nope. Short TTLs are only really useful for switch-overs. This looks like someone's forgotten to lengthen it again afterwards.

          Vic.

  2. hamiltoneuk

    Flakey

    IT: flakey, always has been...

  3. Tony S

    Now working

    I tried a couple of times this morning without joy; finally managed to get access about 30 minutes ago to make sure things are OK. But checked it just now and it's back offline again.

    Unfortunately, my pay has not gone into the account yet, but I suspect that's because they were late making the transfer. Fortunately, I left sufficient in the account to ensure that I covered all the SOs and DDs.

  4. GrizzlyCoder

    Perhaps they should enlist the help of First Direct whom they own and who seem to never suffer any outages

    1. This post has been deleted by its author

      1. Jason 24

        Whilst FDs online system is nowhere near as complete as HSBCs, it does let you get the basics done. Never understood why they can't just take HSBCs code base and reskin it, but then I guess this little incident may give us a clue why!

        Their phone support is 2nd to none however, and you can usually get all the answers you need off the first person you speak to.

      2. Martin
        Happy

        Really?

        I am a First Direct customer and have been for ten years.

        Their on-line facility is perfectly fine as far as I can see. What do you want different?

        Their telephone system is also excellent, with generally very friendly and helpful staff - an exemplar of how a telephone help-desk should work.

        Honestly - I don't work for them - I'm just a happy customer. It's difficult to believe that HSBC own them.

        1. Mark H

          Re: Really?

          Another happy FD online user here. Not sure what other features they could add?

          Certainly not 'tat' and never offline.

          1. AMBxx Silver badge
            Happy

            Re: Really?

            First Direct for personal. HSBC for business. Pretty hapy with both. They leave me alone and I don't bother them!

            Before judging HSBC, we need to know what went wrong. We can also judge them based upon how honest they are about what went wrong - no TalkTalk please!

          2. Can't think of anything witty...

            Re: Really?

            For me, the FD website was pretty weak. Yes, it did most of the things that i wanted it to, but to me the interface felt clunky and poorly thought out. i left them a while back now so i'm struggling to remember all the things that bothered me and there wan't much but it certainly wasn't a great interface.

            nagging issues included:

            -Always trying to open in a new window as opposed to just in a new tab

            -poor website scaling so that when i forced it to open in a new tab it looked cramped and dark (ok - i accept that this is slightly my own fault!)

            -When you log in (or pay, i cannot remember now), you have to click through a screen to select your account, even when you only have one account with them. so you have to select one option from a list of one, waiting for the page to load each time. tedious.

            -the PIN-Sentry thing was a pain.. and you only had limited functionality without it.

            That, combined with some good switch offers from competitors made me swap accounts (which was very simple and easy to do)

            That said, you are correct, i don't recall any connectivity problems. But although their phone service is second to none, i would much rather get things done online as phoning anyone (even the best people) always felt like effort i couldn't be bothered with.

          3. jonathanb Silver badge

            Re: Really?

            HSBC is, as far as I'm aware, the only bank that lets personal customers do international payments online, which can be useful, but does make it extra valuable to phishing attackers.

  5. hi_robb

    HSBC =

    Huge Shower Bloody Clowns.

    1. Anonymous Coward
      Anonymous Coward

      Re: HSBC =

      Helpfully Sheltering Banking Criminals

      1. Graham Marsden

        Re: HSBC =

        Highly Suspect Banking Corporation

        1. JaitcH
          WTF?

          Re: HSBC =

          A US Congressperson described the HSBC as a criminal enterprise.

          And the HSBC's former leader is now in the House of Lords. HOW APPROPRIATE!!!!!

    2. Anonymous Coward
      Anonymous Coward

      Re: HSBC =

      Horrendously Shit Banking Computers.

    3. seanj

      Re: HSBC =

      Holy Shit, Borked Cron

  6. Steve Evans

    Hmmm....

    How much does this smell of a Chrimbo/New Year break upgrades?

    BOFH probably didn't appreciate being told he had to remain sober on NYE and work through the weekend...

    1. Velv
      Boffin

      Re: Hmmm....

      Can't speak for HSBC, but the three other Banks I've worked for have a change freeze from early December usually through to the first working week in January due to the anticipated volume of transactions and the size of the impact if something goes wrong.

      In theory only critical fixes to resolve or prevent issues take place, so anything planned like "upgrades" is a no-no.

      Doesn't mean it isn't the cause here...

  7. Anonymous Coward
    Anonymous Coward

    The problem last year wasn't with Bacs

    It was their ability to submit to Bacs.

    Also it is just Bacs. It hasn't been called anything else for long time.

  8. MyffyW Silver badge

    Woody Guthrie

    "The banks are made of marble

    With a guard at every door

    And the vaults are stuffed with silver

    That the miner sweated for…"

  9. JimmyPage Silver badge
    Stop

    “We will ensure customers do not lose out as a result of this issue.”

    has this weasel statement *ever* been followed up ?

    After the 2010 RBS outage I recall quite a few examples being mentioned where house sales had fallen through because funds were inaccessible (one idly wonders if conveyancing contracts now factor in "if your bank goes titsup" clauses).

    So how far will this statement go ?

    Is there a market for an insurance policy which will make funds available to cover a serious bank outage ?

  10. Dan 55 Silver badge
    Happy

    Every time I read a story about a huge banking titsup happening...

    ... I think to myself how fortunate it is that I bank with Nationwide and carry on with my day.

    Icon is smug face.

    1. Caoilte

      Re: Every time I read a story about a huge banking titsup happening...

      I've never heard of any problems with Nationwide, but do you actually have any evidence that their systems are any more modern than the other banks?

      I got the impression this was an industry wide problem (exactly what you'd expect given the game of musical chairs played at board level)

      1. Dan 55 Silver badge

        Re: Every time I read a story about a huge banking titsup happening...

        I don't know if their systems are more modern or not, but I do know that they seem to have more success keeping them running.

        They have been a bit silly with me on a couple of occasions, but they haven't made any mistake that has costed me money or left me being unable to pay, which seems to count for something these days.

        1. Anonymous Coward
          Anonymous Coward

          Re: Every time I read a story about a huge banking titsup happening...

          NBS are outsourcing their IT (to offshore) as fast as they can; Systems Support, Development, etc.

          I'm sure it will all work out fine.

      2. Anonymous Coward
        Anonymous Coward

        Re: Every time I read a story about a huge banking titsup happening...

        IIRC a couple of yers ago Nationwide moved all their banking IT over to an (unamed) Enterprise system. If that's the case they may be the only bank without "a box in the corner that no-one dares touch because no-one knows what it does", and one of the few large banks whose core doesn't still work internally in £sd with amounts being converted on the way in and out.

        They still have problems with their clocks in the summer, however. Try to execute an online payment "Today", shortly after midnight BST (ie between 23.00 and midnight GMT) and the tranaction falls over. (Gracefully, admittedly.)

    2. Anonymous Coward
      Anonymous Coward

      Re: Nationwide.

      so do I. However I would be careful who sees your smugness. These bunch of cretinous losers caused me a great deal of inconvenience by issuing a new (completely unneeded) switch card[1]. When challenged, they lied about why they had done it, and when *that* was challenged, changed the story. I pushed the whole saga right the way through the Ombudsman, only to discover that since I hadn't lost anything (remember - your time is worthless to a court) they weren't prepared to do anything.

      So now, as far as I am concerned, Nationwide are Just Another Bank.

      [1]Just you wait until you want to park by mobile in the pissing rain, only to discover that you forgot to update the service with your new card details, so you try to do it there and then. Plus the fact that the issuing of the new card started the clock ticking on the old one - if I had been abroad I could have been very embarrassed)

  11. Anonymous Coward
    Anonymous Coward

    Mobile app should be working OK, browser is still call gapping and accepting about 1 in 5 attempts

  12. jeremyjh

    Awful service when it does work

    Online account management for my new HSBC credit card seems to consist almost entirely of a system which triggers the printing of letters. Even Virgin Money's site can take change of direct debit instruction online.

  13. Nick Sticks

    Mobile app marketing campaign...

    I suppose this is one way to push their mobile app but what's the angle?

  14. Marvin O'Gravel Balloon Face

    Is this the same HSBC that announced 8000 staff cuts last June?

    Probly just coincidence.

  15. Anonymous Coward
    Anonymous Coward

    HSBC - The World Local *cough*

    Corrupt bunch of twats...

  16. Anonymous Coward
    Anonymous Coward

    first direct is up.....

    ......for now

    As for the smug AC who banks with Nationwide, don't think it was that long ago they hit a technical pothole.

    Systems break from time to time and banks are no exception and let's be honest, it doesn't happen that often.

    I'd expect the readership of this fine publication to understand that more than most, considering a lot (myself included) work in the IT field.

  17. Howard Hanek
    Holmes

    Banking 'Bottlenecks'

    I always fear that these incidents can be traced to some hacker ring Eastern Europe with the money ending up in some unmarked account somewhere......

    1. Mellipop

      Re: Banking 'Bottlenecks'

      Yes, I wondered that, too.

      I have noticed a number of HSBC phishing emails appearing in my email spam lists. I wonder if they can use that to create a ddos that comes with cookies HSBC front ends recognise?

      Less of the "my bank is better than yours" and more techie analysis.

      1. Anonymous Coward
        Alien

        Re: Banking 'Bottlenecks'

        Rumours abound that some chinless wonder on "work experience" accidentally "upgraded" the whole system to Windows10.

        .

        .

        .

        .

        .

        .

        .

        .

        .

        .OK, I made that up.

        .

        .

        .

        .

        .

        .

        .

        .

        .

        .

        .

        Or did I??

  18. DBAosaurus

    Entirely predictable

    None of this should come as a surprise. Retail banks present a glossy and slick exterior to their online offerings. But from direct experience of those where I've held contracts as a DBA, the underlying systems and software are dated, poorly architected, understood, maintained and documented.

    These systems are badly developed and tested and held together with pieces of string and chewing gum. The innate conservatism, internal bureaucracy and short termism of banking culture generally prohibits any desire to grasp these nettles. Specifically systems development and testing is invariably outsourced to large teams of incoherent offshore contractors, most of whom struggle with basic literacy and clarity of expression and have no concept of method, data governance and quality, programming technique or how to actually write a performant SQL query.

    The whole shambles is then nominally under the control of layers of dysfunctional management, many of whom are simply not up to it. But they can count beans and tick boxes. Expect similar borks from HSBC's competitors this year.

  19. JasonLaw

    The world's local bank

    I'll just pop into my local branch instead. What? They closed it 6 months ago?

    Amazing.

  20. Anonymous Coward
    Childcatcher

    Thanks for the photo

    My wife is obsessed with purple, and now she wants a purple house phone!!

    There is only one purple thing she is not keen on.................

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon