Sure, the games move you - but what you *feel* is the controller...and if the controller feels right it will help your immersion.
I can see where you're coming from better here. For me, the perfect feel for a controller is none at all. If it's well designed and laid out, it should become instinctive, invisible. When I play an FPS with mouse and keyboard, these days it's instinct. The control set works well enough that I simply *think* to look to the right, and the process inbetween is (at the time) seamless.
It's the same with a mobile device, or a computer, or any other electronics I use. It should provide a good base set of tools, a solid set of hardware, and then get the hell out of the way. My love of the N900 wasn't based on it being slick (though it was, in its own way), but more on the sheer brilliance of the available toolset that let me get on and do really cool things with minimum effort.
We're probably coming at the same thing from different angles, but with me devices are 100% about functionality. Looks, feel, and "soul" come waaaaay down the list. ;-)