Re: just goes to show how little ...
While the radiation produced directly from the explosion is relatively short lived, it'll still do a (considerably big) number on you, and you won't be living much longer than a few hours or days, depending on proximity to ground zero (Which, in the case of these rockets, you have a few miles of atmosphere between you and the detonation, so that's all the alpha particles and likely almost all of the beta gone too, so only a minor gamma burst to deal with... and none of what I'm about to write about next...)
The biggest issue is the material that gets sucked up as part of said explosion, and irradiated to produce the real fallout that goes on to cause much more long term damage than the initial explosion. This is the stuff that will prevent you from going back there for a good amount of time, or more likely cause long term problems and issues due to people going back and not understanding why it's so dangerous. Lots of those who went back to help those injured in the original bomb blasts in Japan succumbed to the secondary fallout radiation without knowing it
Not only that, but this material can spread in winds and rain, this was the major issue surrounding Chernobyl and the massive cloud of debris it produced, which was capable of scattering radioactive fallout for many thousands of square miles, potentially endangering huge amounts of the nearby population, even drifting towards Europe and causing havoc there.
The damage you're imagining is more than you think. Most wouldn't think dust and the like floating down would be harmful in that way, and by the time you know about it, it's too late.
As for your last point, obviously, it's hard to talk to anyone who has died from radiation, because... well, they're dead. Possibly buried in lead lined coffins too. Not hard to find though (not like they're going anywhere), just takes more effort than you want to put in because it's easier to make false blanket statements about things like this, contrails and moon landings than it does to simply state what you think.