back to article US taxman slammed: Half of the IRS's servers still run doomed Windows Server 2003

Half of America's Internal Revenue Service's (IRS) servers are running Windows Server 2003, despite extended support for it ending in July. That's according to a report by the Treasury Inspector General that took a look at the IRS' $139m upgrade program. The report is distinctly unimpressed and notes that the IRS "did not …

  1. Chris Miller

    While it's clearly not good to be still running Windows 2003, these servers are only "wide open to hackers" to the extent that they're exposed on the Internet, which one would hope they are not. Yes, if hackers can get malware inside the perimeter, outdated servers make their job easier; but if that happens you've already got a problem.

    1. Dadmin
      Thumb Up

      I see you too subscribe to the classic blunder of the Eggshell Security Platform, wherein it is written; None Shall Pass Our Firewall Of Justice, unless you get by, then have at these outdated, unpatched servers that no one bothers to secure locally because; firewall, guy. The idiot I work for thinks that by never updating our internal servers we'll never get another new bug! He also thinks that strapping everything onto NFS is a surefire trip to the top of the executive food chain! Let's not bother making solid, custom VMs, let's just strap on some NFS dirs for everything from apps to monitoring agents (only on NFS, did I mention that?) and call it a day! Oh, did I mention we should never upgrade our 5 year old servers? That's worth bringing up again. The guy must have been an admin for a few months and reads too much CIO magazine, because these "solutions" he dreams up are shit. Every single one. I always make it a point to ask; "where is the security officer for our company?" which is met with strange looks and no answer. Come hack this place, it is a malvertizing calamity waiting to happen.

      Help yourself! Also, please do US all a favour and erase all data for citizens who make less than $150K/yr US. Thanks!

      1. a_yank_lurker

        What the -220V USB sticks being littered in IRS parking lots?

  2. Ugotta B. Kiddingme
    FAIL

    our tax dollars at work

    except, you know, NOT...

  3. Anonymous Coward
    Anonymous Coward

    Priorities

    Probably too many resources were thrown into persecuting those who dared to oppose Obama in the last couple of elections. Okay, the IRS might get hacked and so forth, but that won't degrade their (current) primary mission, which is to silence conservative groups when it really counts.

    1. Dan Paul

      Re: Priorities @Bigjohn

      And EVERY position in the Obama administration was filled with Obama's appointees over the last 8 years. Notice how many military leaders he's burned through?

      Therefore, EVERY problem that you hear of with the USA IT systems ultimately falls at the feet of the guy who looks like he should be the black Alfred E. Neumann of Mad Magazine. IE Dumbo the clown. How appropriate! What, me worry?

      Obama's appointees hired or oversaw the very people that are supposed to maintain and secure this nations IT assets. That makes this ALL his fault right down to running insecure email servers for primadonnas, letting the Chinese and Russians run roughshod over all the OPM data. And anything else they took with them while they were rooting around like pigs in our servers.

      Burn Lois Lerner and feed her to the employees of Goldman Sacks.

      You don't want to hear my feelings about our so called President!

    2. Anonymous Coward
      Anonymous Coward

      Re: Priorities@ Big John

      How dare you criticize Obama on this site! /Sarc/

  4. hplasm
    Happy

    Sympathy where it is due, after all...

    Not here, I imagine.

  5. Cincinnataroo

    Unusual project approach

    It might be that avoiding the "usual approach" to project management has been beneficial. Steering committees and reams of paper may slow projects down. Maybe by taking charge without bureaucratic millstones has converted more servers than would otherwise have happened?

    Would be good to hear from a worthwhile technical source, rather than some folk who may be paper pushers.

    1. Tom 13

      Re: Unusual project approach

      While there is much truth in what you say, there are problems too. I've been on the receiving end of one of these "push it through now because it should have been done 5 years ago but the committees got in the way" projects. In an office with around a 1000 people we had hundreds who lost access to files as part of the migration. Mind you we weren't doing an upgrade, just a double hop network migration (current network had the same network as the parent network we needed to join so first it had to be renamed). Yes, we sorted it all out by the end of the week and no data was lost. But it would have been better had some appropriate planning been done.

      As always the trick to getting these things to work properly is having a committee just large enough to cover all the details and have everybody on the committee committed to achieving the goal. Sadly that's one of the most difficult things on this planet to accomplish.

  6. TidySweep

    Microsoft is Unrealistic

    I really think that Microsoft is unrealistic about server software EOL. These businesses put the machines in place and intend to run them indefinitely. It's not like Windows - Easy -Transfer is an option, because it is not. When Microsoft sells an OS to a big corporation, the Microsoft folk should do so well aware that there's no easy upgrade. They should be thinking in terms of quarter centuries (and in some scenarios, longer).

    1. localzuk Silver badge

      Re: Microsoft is Unrealistic

      It isn't Microsoft that is being unrealistic, it is the organisations who think they can run servers indefinitely.

      The bottom line is this - Microsoft is a business. To survive and thrive, they have to innovate, release new products and earn profits. Expecting them to support an OS forever is just silly.

      It also ignores the realities of IT - things change. New systems improve on many of the old issues that organisations face. I mean, I'm chomping at the bit for the new Windows Server OS to be RTM'd and become stable. Some of the new features will save this organisation a chunk of cash, and save the IT team a chunk of time.

      No, organisations should pay attention to what they are buying. No OS is going to last forever. Microsoft have been pretty clear with lifecycles of their OS's too, so there is pretty much no excuse.

      If a business thinks a product that they bought 25 years ago is still serving them well, then they more than likely are going to be struggling to keep up with the modern world they are trying to sell to.

      1. Chika

        Re: Microsoft is Unrealistic

        To an extent, you are both right. And wrong.

        Treating the OS of a server in the same way as you treat a desktop OS is unrealistic on the part of both Microsoft and the users because a server is used in a completely different fashion. Moving from one OS to another can take a lot of effort, not just on the part of the OS developer but the developers of any software related to that server. OK, such simple items as file server and print servers are pretty simple to sort out but even then a wrinkle will occur that can stop a migration dead in its tracks.

        I don't necessarily believe that an OS for a server needs to be supported indefinitely but having hard defined cut off dates can be a problem for any operator if an application or a driver just isn't there. Or indeed the budget for whatever replacement work is needed.

        There also appears to be a habit of people assuming that an OS becomes a liability (or even unusable) the moment that support stops. As even Microsoft will admit, part of the security of any OS is related to the way in which the system has been set up and while they can advise on that, if it is done wrong then a Windows 2012 Server can be as much of a liability as an NT4 box. W2K3 isn't going to be a major problem as long as it was configured correctly in the first place, though I agree that it isn't something I'd leave in a sensitive area indefinitely.

        The question, therefore, isn't that the W2K3 installations are there, but whether they are doing anything about it. (Of course, as someone else pointed out, one solution is to pay for the extra support...)

  7. Kev99 Silver badge

    Gotta love software vendors. Unlike automobiles, motorcycles, televisions, appliances, etc, if you service for a 1914 Baker, you can still get the parts.

    1. localzuk Silver badge

      From the original manufacturer? A quick search would indicate that after a couple of takeovers, the company ended up going bust.

      So, you're talking about copies and after market goods.

      Cars aren't the same as complex software systems consisting of millions of lines of code...

      1. Tom 13

        Re: From the original manufacturer?

        No, the difference is the OMs for cars did't get copyright protection for their parts like M$ did. I expect that if MS IP protections lasted only as long as they do for autoparts, we'd have a healthy ecosystem of third party vendors keeping DOS alive and well.

        Oh, and when you are paying as much for a new server as you would for a car, it damn well ought to last at least as long as the car would.

  8. Kev99 Silver badge

    Gotta love software vendors. Unlike automobiles, motorcycles, televisions, appliances, etc, if you need service for a 1914 Baker, you can still get the parts. Not so with software or computers over three years old.

    1. Sureo
      WTF?

      Microsoft discontinues support on a wildly popular and perfectly adequate product, resulting in huge expenses for their customers, because they can't be bothered to fix their bugs anymore?

  9. Benno

    The mind boggles on how poor their ICT systems must be for them to not know where over 1000 XP boxes are. While they may have a massive fleet (I expect it would be in the hundreds of thousands of systems), surely they can use one of the multitude of auditing methods out there to find those machines?

    If they're on an AD, it's a no-brainer - even if they're not, it's still straightforward...

    1. Roland6 Silver badge

      Re: auditing method

      "straight-forward" auditing methods when dealling with 10,000's of systems across multiple sites, only really help you identify the live machines connected to your network...

      No at some point they are going to have to make a decision based on how confident they are that none of these 'missing' XP systems are still 'live' (and that includes the laptop in the bottom of the Security managers draw that was used to generate certificates) and thus write these systems off and so stop paying MS the annual licence subscription.

      My guess is that at least a few of these systems are being used by employees/ex-employees in their homes. With the majority having been replaced (and hence scrapped) without inventory records being updated.

    2. Tom 13

      Re: The mind boggles

      No, it's worse than that. While not at IRS I've seen some of the things the government does at my agency. I try to block it out because if I left my mind think about it for more than a minute or two it would quickly be reduced to ooze.

      Mind you, on the tech side, I'm in a pretty decent shop at the moment. We actually complete our quarterly scans every quarter (95%+) and we get an actual touch on all the mostly disconnected systems. Last place I was at religiously did their quarterly scans once a year.

  10. Anonymous Coward
    Anonymous Coward

    been there, done that

    Did that in a manufacturing plant, where a large number of machines were locked in cabinets, in clean rooms, etc. where physical inventorying was contraindicated.

  11. Christian Berger

    One should note that it's precisely the same in most companies

    It's just that in the IRS, we get to know about it.

    Of course the underlying problem is the same for all organisations. In the early 2000s they bought from Microsoft believing that they would continue to develop Windows in a reasonable way. Instead they got XP (which needs more than 2 Gigabytes for the OS alone!), Vista, 7 and 8, as well as their equivalents on the "server" front. All systems which don't offer any useful functionality, but require new hardware and introduce new incompatibilities.

    1. Roland6 Silver badge

      Re: One should note that it's precisely the same in most companies

      " In the early 2000s they bought from Microsoft believing that they would continue to develop Windows in a reasonable way."

      And here we have the real crux to the problem and the problems business now face with MS and the looming EoL for Win7 et al. There is little confidence that MS are and will continue developing Windows (or their other enterprise products) in a reasonable way. Company's such as IBM on the other hand, have demonstrated with their 1960's mainframe technology their capability for reasonable development.

      Unfortunately, I do get the impression that the open-source movement hasn't (yet) fully understood and grasped the significance of this concept.

    2. Tom 13

      Re: One should note that it's precisely the same in most companies

      No it's not. For one thing most companies aren't immune to federal regulations like the agencies that enforce them are.

      Even in government most agencies have done a better job than the IRS. My own agency is mostly running Linux servers and the few Windows boxes are mostly 2013 with a few stragglers at 2008. The desktops have been Windows 7 SP1 since I got here three years ago, I think the upgrade was finished the year, possibly two, before I arrived.

  12. Anonymous Coward
    Anonymous Coward

    'Cause I'm the taxman, yeah I'm the taxman

    It's only (other) people's money!

  13. bitmap animal
    Meh

    It us still supported.

    Am I the only person that read the line "In an effort to avoid a massive security breach, the IRS has agreed to pay Microsoft an undisclosed "premium fee" to continue to support and patch its servers".

    They have an infrastructure that works, or at least I presume it does, so the additional costs for the continued patches and support will be an element in the whole company IT cost. Changing to a new OS may well mean different platform versions and so many of their systems would need changing and testing. If it ain't broke, don't change it.

    1. Tom 13

      Re: If it ain't broke, don't change it.

      While I do generally subscribe to this maxim, in this case not having the same support as the primary OS does make it broken. While you may be paying for patching and support, since it isn't getting the same support as the OSes they are selling, you're still at a significant disadvantage. In my estimation, this only reinforces the need for an industrial grade OS where you can expect support for a 20-30 year period of time.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like