"I realise you can't beat the the novel format after all."
I think the real truth is that you can't beat the *original* format. Each format has strengths and weaknesses and a competent author will play to the former, making it very hard to produce an equally good "translation" into a different format.
As a result, producers who buy the rights to a book might be better advised to buy the rights to the characters (or universe) depicted therein and then commission someone to write a different story. (This needn't be the original author and unless that author has some experience in the other medium perhaps it shouldn't be.)
I think I read somewhere that JKR made a lot of friends when she first met the producers of the first film and suggested that quite a few things would have to change to make the books filmable. Cue huge sighs of relief from the screenwriting team.