German net crippled by top level glitch
Not hard to do
I assume DENIC uses the BIND nameserver software. If so, this is an easy error to make. I'm not excusing DENIC, just observing that the scale of such a problem is often far out of proportion to the cause.
BIND parses a zone file as it loads it, and ceases loading (that zone) after it encounters an error. This could be a syntax error or an invalid use of a record. "Invalid records" are not always obvious: it's easy to accidentally create a host entry within a zone that was already delegated out.
Unless their update process (presumably either Perl scripts or a product like Men and Mice) correctly validates the entire zone *using the same criteria that BIND is configured to use*, errors can slip through.
I used to work at a large Internet site, and have pushed out more than my share of broken DNS zones. Good tools[0] help immensely, and a good architecture[1] insulates certain problems from reaching the Internet. None of those achieve perfection, though.
The only way to reliably catch this sort of problem is to carefully test all changes in a staging environment first. In the end, it's a balancing act between the time required to perform all due diligence and the quick turnarounds (and in this case high volume) demanded for DNS changes.
[0] The Men and Mice tools revolutionized how we managed DNS.
[1] For instance, push changes to a hidden master and let that master propagate to your main (slave) nameservers. Monitor the bejeezus out of the content on that hidden master. Run scripts that parse the named error log and immediately alert if there's a potential problem. Et cetera.
