it's all about the video, of course.
My ideal TV would copy 5.1 sound systems, where you plug in a microphone and they can make a passable effort at self-calibration. Why can't TVs do this? Most people i know are watching their TVs with colour and contrast completely out of whack.
How about a very rapid power saving mode that goes off when there is no one in the room and comes back on again when you come in? Audio muting optional.
How about non-sucky picture-in-picture controls? Or digital logo removal processing?
There's a lot of smart things TVs could do, but smart TVs don't seem to do any of them.