As a relatively happy Galaxy Note (CM10.1) user I wish I had a pound for every time I've heard the Dom Joly remark!
Personally I'd quite like something just a tad larger. I tend to use the device for video, reading ebooks, gaming, nav etc so an even larger screen really appeals. When I do use the Note as a phone I've never get any major problems from that side of things.
As a custom firmware flasher though I have to say that Samsung's exynos is a bit of a mess really. They've really not given the opensource code that the devs need to make it a great custom fw platform. Sony on the other hand have been extremely helpful and supportive. That's why, unless something even better appears in the meantime , my next device will be the Yuga. The only thing I'll miss will be the S-Pen since that makes texting/mailing/form filling a brilliantly fast process.
I guess the point is though that Samsung have pretty much comprehensively shown that there's a market for large form factor smartphones. The battleground is your pocket! Just how big will manufacturers dare to go? 6" is about right for a pocketable HD media device.