"Absolutely ludicrous. What's to keep me from cobbling together a crystal transmitter device that broadcasts on whatever frequency I want it to?"
On licensed bands, the FCC vans. They'll hunt you down and issue a citation.
But they did create these ISM bands with exactly people like you in mind. Within limits (out-of-band radiation, etc.) you are free to do what you want. Induction heating, ground penetrating radar, RC toy control, etc. Its all good. I suspect that the initial R&D on weather radar used an ISM band for exactly this reason (the S in ISM). And that's OK, right up to the point where you want to put the radar into 'production' use. Then, it would have been better to assign a nearby licensed frequency for its operation. Where the FCC has the authority to slap down violators and hasn't already let a few million non licensed devices out onto the market.
Within licensed bands, the FCC has the names and addreses of all the users. And they can reassign frequencies without much problem. But once they let a block go as ISM, it's really difficult to pull it back. There are tub skids of 5 GHz WiFi hubs lacking firmware lockdown available at the local PC recycler. Enough to keep the firmware hackers supplied for years.