I agree that it depends when you first encountered the show.
Being the age I am, Davidson, Baker (Colin), and McCoy are my Doctors - I have a particular fondness for McCoy, but that might have more to do with Ace and my hormone levels at the time. Regardless, I always enjoyed watching repeats of earlier series and it was always The Doctor, no matter which actor was playing him - I don't think they've ever miscast.
It's a crying shame that McGann never got to do any TV episodes, as he's a fantastic actor, and an excellent Doctor Who (judging by his audio work, and the recent mini episode). Thanks to the extend hiatus, the new Doctors just aren't my Doctors - my kids are just getting into it, now, so I'm hoping that will change.
Is there an icon for terminal fence sitting?