Wow. I set up in consultancy five years ago and I didn't bother with half of that crap.
You do the minimum you can to get by - incorporate (aka form a Ltd Company), register for VAT, get an accountant, join PCG, buy employers liability and public liability insurance, (professional liability insurance if you want to go belt and braces), done. About £500 down and you're a consultant.
My (ex-)wife (still) does the books. I don't need accounting software. And why would a one-man company need Salesforce.com? Notepad has most of the CRM features I need.
I bought a domain and had a vanity email address - that was a waste of time; clients are quite happy (even prefer) to use my personal email address for comms.
As for VoIP, I have 600 minutes on my mobile anyway, and wow look my printer is also a fax machine, who knew? Again, clients *like* having my personal number.
I think the main thing when starting up a company is to defer anything that's non-essential in favour of, you know, doing actual work for people and earning actual money. So maybe I'm not in compliance with some obscure piece of employment legislation. So sue me.
YMMV. I'm a programmer/architect. If you're a sysadmin maybe it is important to have a more impressive backoffice; and if you're the paranoid type, maybe it's important to cross every "i" and dot every "t".
The one thing I did which was unusual was to hire an agent to get your first gig. Sure, they take 15-20%, but they get you your first gig. Do that, and you have a satisfied ex-client, and you're on a roll!
Don't skimp on the accountant, though. I'd definitely agree on that one.