@AC -- The GUI is your friend
I work for a large state government in the U.S. and I've been using NetBeans to write a pretty complicated desktop application (financial reporting, mostly) that we'll roll out later this year. Java's a really nice choice for doing desktop software, and NetBeans takes care of all the Swing plumbing for you. You still have to know how to write the code for your event handlers, database interfaces, PDF generation, etc... But if you can't do that, you're not a java programmer, right?
I don't do applets (shudder) but for desktop software, Java's great. As long as you're careful about your memory use, it won't even beat up your PC that much. I don't even worry about "write once, test everywhere" because testing is kind of fun. You run it on some crappy old PCs, some macs, some modern laptops, try to make it break... It's amusing and you can have some popcorn while you try to break your app with wonky inputs.
As far as certification goes, I'm divided on the issue. First of all, it won't affect my job in the slightest. Nobody in government cares about how many pieces of paper you have; you're interviewed by a team including the lead developers, some managers and usually a system admin, and if you can't satisfy them you don't get hired. Anything that makes them nervous or suspicious disqualifies you instantly.
That aside, studying for the programmer test could improve your skills. It seems as though they ask some pretty arcane questions, so you'd have to make a pretty thorough study of how the system works to pass. More knowledge of the guts of the system can't be a BAD thing.
When it comes to the later certs, though, aren't they really pushing a development philosophy? That stuff changes with the wind in IT circles. All you'd be proving is that you're up on the latest fads (web services in this case).