back to article Transport for NSW scrambles to patch servers missing fixes released in 2007

Around a third of servers at Transport for New South Wales, the public transport department in Australia’s largest most populous state, need security patches, some dating back to 2007. But IBM, which provides IT services to the agency, doesn’t have enough people dedicated to the job to get it done in the planned time frame or in …

  1. Likkie

    Whilst NSW does have the largest population, it is in fact the 5th largest state/territory.

    1. Anonymous Coward
      Anonymous Coward

      NSW

      Is that according to the recent Australian census cockup, run by, erm, IBM?

      1. Likkie

        Re: NSW

        "Is that according to the recent Australian census cockup, run by, erm, IBM?"

        Now there's a mouth full of coffee I'lll never get back ;) .

        1. Anonymous Coward
          Anonymous Coward

          Re: NSW and coffee gone

          Until you read the next paragraph (assuming you swallowed it dramatically)

      2. Laurie_down_under

        Re: NSW

        IBM will never throw the customer under the bus, even though many of the issues with the census were their fault. Effectively, they DOS-ed themselves by telling everyone in Australia to submit their census at the same time.

  2. as2003

    I seem to have read a lot of articles that contain words to the effect of "IBM declined to accept responsibility, stating that...", but it's always IBM. Hmmmm.

    I wonder if any redundancies have been made in the sales and legal departments. I doubt it.

    1. Anonymous Coward
      Anonymous Coward

      So much for RESPONSIBILITY

      This IBM is the same IBM that knowingly sold Microsoft operating systems on it's PC that had known faults on them by Microsoft. Microsoft refused to fix them saying IBM signed an agreement to take responsibility for them, but never fully handed over the capacity to deal with those specific faults, so IBM were unable to fix the faults.

      Australian consumer LAW, requires both the retailer AND Manufacturers to assume responsibility for their product being sold within Australia.

      Both were responsible for the shitty situation (an argument between IBM & MS) IBM told the customer(user) to go to Microsoft, Microsoft told the customer(user) to go to IBM.

      FAIL, FAIL, FAIL - people are still working in IT that allowed this to happen, in higher places than where they were when it occurred.

      1. Anonymous Coward
        Anonymous Coward

        Re: So much for RESPONSIBILITY

        Maattteeee, IBM hasn't sold Windows PCs since 2004 ! It's a stretch to blame IBM for selling the hardware when it was a Microsoft OS.

      2. G2

        Re: So much for RESPONSIBILITY

        given that IBM is dealing mostly with companies (or should i say "exclusively"?), i'd say that is standard behaviour in Europe.

        You are lucky to have such a consumer law in Australia that allows companies to be considered as "consumers" because here in Europe you're generally fucked if you have a consumer-type issue and you are not a natural person. :(

        Under EU law, the notion of consumer does not extend to legal persons (companies), even if they have a non-business character (e.g. non-profit associations). So, if it's not directly written in a contract then it doesn't exist. There are some minor differences from this in some EU countries but generally that's the rule.

        https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:31993L0013

        quote: ‘consumer’ means any natural person who, in contracts covered by this Directive, is acting for purposes which are outside his trade, business or profession; /quote

        Not even self-employed traders or family businesses can be considered as "consumers" - but there are, again, some minor differences to this rule across EU countries.

  3. Chris Miller

    I thought transportation to NSW ended in the 1860s ... oh, sorry, misread the headline.

  4. Anonymous Coward
    Anonymous Coward

    Meanwhile in a parallel universe, the exec(s) who made the initial screw up in deciding to cut staffing to the point that this basic kind of cock-up could occur, and who got nice fat bonuses for saving costs, were held responsible for their errors and had their bonuses taken back off them and told they're incompetent.

    Not in this universe though...

    1. Dan 55 Silver badge
      Devil

      Don't you just love these multinationals that pocket billions and behind the scenes on every project it's two or three overworked staff barely keeping the thing going, while marketing just sold the cure to cancer which will be ready in six months and suddenly staff need to be taken off other projects to come up with that too.

  5. Tim99 Silver badge
    Windows

    Eh?

    "as while offshore labour will be involved it can only do so much when on-premises mission-critical servers require reboots. " Well there's your problem. I'm so old, I can remember when IBM's staff were all locals, and the only offshore people your organization saw were very senior specialists that they occasionally flew in...

  6. JeffyPoooh Silver badge
    Pint

    "Servers require security patches"

    Notice: "Server will be off-line tonight from 8PM until approximately 5AM."

    Click-click. Click-click. Click-click.

    Coffee.

    Click-click. Click-click.

    Yawn.

    Click-click.

  7. Anonymous Coward
    Anonymous Coward

    Oh, NSW.

    Oh, NSW = New South Wales

    The only reason that I opened this article is because I thought it was NSFW.

    .:disappointed:.

  8. Anonymous Coward
    Anonymous Coward

    In IBM's defence...

    The customer brought the name and not the service.

    How were IBM to know that laying off all the local staff to employee cheaper foreign staff would only allow them 3-5 years of cost savings before they had to layoff the foreign staff and find someone cheaper?

    Or that laying off experienced staff would result in inexperienced staff not doing the job o the same standard that the customer used to expect?

    And IBM were cheaper. Well not actually cheaper, but cheaper than the price that they initially wanted to charge the customer before the customer said "are you shitting me?"

    It was a perfect storm really...

    1. Anonymous Coward
      Anonymous Coward

      Re: In IBM's defence...

      They had locals on the account, customer decided they want delivery entirely offshore to save $$$ and pushed IBM to cut local asap. So work was transferred per a schedule and locals were cut per the schedule and to meet staff cuts decreed by Singapore. Problem being that schedule slipped, offshore resources needing to be trained as most were newly minted freshers straight out of school but the cuts locally were made anyway. Factor in attrition of people leaving before they got kicked to the curb and they were down to the point where any absence would have an impact on delivery if offshore teams weren't able/unwilling to take on the work.

      Offshore teams (more than just India) didn't step up to take up the work for which they were engaged and the usual excuses provided - not enough Knowledge Transfer, unfamiliar environment, don't know tools, need to hire more staff - not uncommon to see 2-3 times as many offshore staff hired than there were locals on an account and delivery still worse than local teams who had already been decimated.

      Other accounts are in similar situation - cut so much locally that there are no free resources available to help out other accounts - most local teams can't cover workload if someone is sick, goes on annual leave - work is prioritised based on who will scream the most if it's not done (or whichever has the highest financial penalty.)

      Factor in tight change windows (4 hours in some cases (including customer post implementation validation and testing)) and inheriting systems that had either never been patched or patched inconsistently and barely having enough staff to cover normal activities it doesn't take much for it to all fall apart.

  9. Denarius

    and the NSW PHBs who thought outsourcery was a good idea ?

    none of them given a cut in bonuses either ? In any other industry except electricity generation such behavior would have managers in front of fraud courts.

  10. Anonymous Coward
    Anonymous Coward

    Better off

    Most updates are just buttfucks anyway.

    All are better off with a complete replacement.

    All code contains holes and incompetencies,

    Australia LAW, Fit for purpose applies !!!!

  11. jonnyo

    You mean Watson didn't magically stop this, just like the commercials on TV tell us it will? I guess it was using too much CPU to calculate world peace, the end of global warming, how to find a Mrs. Watson, etc.

  12. lglethal Silver badge
    Stop

    I call bullsh%t...

    "The problems at TfNSW seem to have come about in part due to Meltdown patches throwing other plans out of kilter. "

    "Around a third of servers at Transport for New South Wales, the public transport department in Australia’s largest most populous state, need security patches, some dating back to 2007."

    If you have unpatched servers going back to 2007, the problem is most certainly NOT caused by whatever the latest OS/Chip/general security blunder is.

    1. Anonymous Coward
      Anonymous Coward

      Re: I call bullsh%t...

      I think they mean urgent patching for Meltdown messed with the schedule of remedial patching. Remember IBM hasn't had these systems for long; they're not the once who created this mess in the first place.

  13. Doctor Syntax Silver badge

    Surely IBM has multiple layers of management.

    And surely all those managers are their because of their technical competence - aren't they?

    So surely those managers could lend a hand to do essential work.

    After all, no business would be so foolish as to cut the staff who know how to do the work the company depends on and leave it overstaffed by those who don't. Would it?

    1. Anonymous Coward
      Anonymous Coward

      In IBM, there are two types of technical managers:

      - the ones taking credit for the hard work of others

      - the ones pushing someone else under a bus when something goes wrong

      Note that being a hard worker that others are taking the credit for and being thrown under a bus are not mutually exclusive.

      Between resource actions and managerial incompetence, it's lucky that IBM has such an effective financing operation going on otherwise they would probably have disappeared already.

      1. peterjames

        Is that not the new norm of most liberal capitalist, ehm, neo-feudal, business?

        No investment in or recognition of knowledge or skill, full taking of credit for any results, and the feel of bus rubber over you as part of the daily grind?

        Because enlightenment only happened in Europe, right?

  14. Laurie_down_under

    patching windows

    It's almost impossible to get approved patching windows at TfNSW, and IBM's offshoring certainly hasn't helped.

  15. FozzyBear Silver badge
    Flame

    No updates since 2007. Nice it's not like these systems, control switching on the rails, traffic and monitor progress of trains across the network. Timetables, Comms and maintenance schedules.

    Oh wait....

    Interesting considering they invested heavily to revamp the the systems that log lateness of trains at various stops and the associated reporting. Anything short of a derailment is now considered Act of god and therefore outside of their control. Wouldn't want to interfere with the directors and senior executives getting their yearly bonus now would we?

  16. CFtheNonPartisan

    IBM has more ownership of failures across government than any single company has a right to own, yet instead of being barred they keep getting contracts. Therein lies the problem - procurement failures. You cannot blame your customer for your own failures that you should have had the capability of working out prior to the fail - unless you did not understand your own business vis a vis the customer requirement.

  17. Anonymous Coward
    Anonymous Coward

    TfNSW - NSW Government Jobs for Offshore Workers by IBM

    The NSW government should be ashamed for signing off on allowing IBM Australia to Offshore most of the the support roles for TfNSW to cheap low cost countries at the expense of local jobs. The account and TfNSW has had it's fair share of major issues lately without the need to Offshore to unfamiliar workers. Many local Aussie roles like server OS teams have been targeted to Offshore.

    When is someone in Labour going to call out the Liberals for allowing this to happen.

    IBM sells its capability to draw on it's Global Work force but on a Local account like TfNSW it struggles to support. Keep the Roles in Australia before it's too late. Bow your heads down IBM Australia and NSW TfNSW for this situation continuing.

  18. Anonymous Coward
    Anonymous Coward

    I don't mind beating up IBM, but they're basically being blamed for not being able to clean up someone else's mess fast enough. I'm more interested in knowing who exactly let the servers get into this state in the first place.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019