"Looking ahead, you'll get to worry about losing your physical hardware key rather than losing the secrecy protecting your passwords through a poorly secured server."
You'll also have to worry about the H/W key failing. You'll have to hope that the next computer, phone or whatever still has the slot that your key requires. It's not as if manufacturers would ever decide to eliminate connectors is it? And what happens when you're sitting on the end of a KVM switch with only a keyboard and mouse?