Management is mediocre
It's pointless to say "I can buy xxx hardware for $$$". You probably can, but you have no idea what the scope of these projects are. If you take a agency like the US Department of Homeland Security (DHS), it would be easy to have 500,000 computers involved in a system wide update. How many of those would need to be replaced to be compatible with a core systems update or consolidation project? As a system grows larger it becomes more difficult and less efficient to manage. Ironically, exactly the opposite of the stated goals of combining already bloated agencies into DHS. The Air Force, Treasury, and Interior aren't the size of DHS, but I'd bet they are in the range of Fortune 50 or Fortune 100.
But even on a smaller scale, licensing and labor are the main components of an enterprise solution. A company I am familiar with budgeted $4m to install an ERP. About $100k of that was upgrading hardware. Last I heard they had passed the $6m mark and were well on their way to $10m. And they they only had around 1000 employees.
The scale and scope of many government projects are well beyond COTS solutions, which is why it's encouraging to hear someone at Kundra's level talking about breaking projects into more manageable sizes. I have seen it many times during my IT career; management setting some goal that is so large that staff just sits there with a 'deer in the headlights' look on their face trying figure out where to start.
And COTS worked so well for the US government, they are now suing Oracle for overcharging them. COTS isn't a silver bullet.