Additional comments from Roger Thompson at AVG
For some reason, this information didn't make it into the story:
The change from 1813 to SV1 was part of a planned release. Software can't be changed overnight, but we do have a "hot fix" coming up that will mitigate some of the extra traffic without impeding our ability to protect our users. We're also continuing to gather data, and work with webmasters and analytics folks, and we still enable those webmasters who want to filter our requests out of their results to do so.
In the meantime, the Bad Guys are continuing to improve their ability to mass-hack websites. The problem is incredibly transient, and as fast as websites are cleaned up, others are hacked to replace them. And they're not just minor websites. There are recent examples of security companies, government sites, and banks that have been hacked. Real-time crawling is the best way, from a user protection point of view, to safely discover which websites have poor enough security that they've been nailed.
With Search-Shield, we're not trying to block those websites... that's the job of the Active Surf-Shield component. Search-Shield just shows users which sites they should avoid, on the basis that, if a site's been hacked once, it's typically been hacked multiple times before the hole gets plugged, and some of those other hacks might well contain zero-day exploits. I wouldn't visit any website that we show a red verdict for, except on a goat pc.