My fave was Tom Baker (funnily enough watched The Android Invasion and The Brain of Morbius over the weekend) as he was the incumbant Who during my youth. So he is the benchmark for me. I didn't mind Peter Davidson but admittedly I didn't see much of Colin Baker (who I voted for) and Sylvester McCoy.
Going backwards I wasn't a great fan of Jon Pertwee because he was stuck on Earth (booring) and because I was constantly reminded of Worzel Gummage! I haven't seen too much of William Hartnell or Patrick Troughton.
Of the reboots, I don't have too many complains about the Doctors themselves (not sure about Matt Smith though), but it's the companions and how they're written in that generally have spoiled it for me. I refused to watch the series when Catherine Tate was in it. For me there's too much touchy-feely/will-they-won't-they stuff going on, and as other commentards have pointed out, this is meant to be about Dr Who, not about the domestic lives of his companions. Also if you have a craft that can pretty much go anywhere in time and space, why does the TARDIS seem to keep going back to South London?
I probably will watch the new series, but I can't say I'm overly excited about it. I just wish we could go back to companions being just that, and not some sort of love interest.