> And I'd have thought the authorities would have some pretty good web crawler technology, so they can find all this stuff by themselves? Also, the DNS part only gets you the server, not the URL, so unless the site has an obvious name like dodgy-images.com how are they discerning content from DNS lookups?
So, to answer your question, this is how the system works (or at least used to). This is based on BT's Cleanfeed, which was the original implementation (and the one first misused to block torrents as well as child abuse material).
Lets assume example.com/fine is legal and above board but example.com/secret contains illegal content.
- You try to visit example.com
- Your browser does a DNS lookup for example.com, which is intercepted by your ISPs DNS servers
- The ISP uses the IWFs list, and finds example.com on there
- They return you A record 1.2.3.4
- Your browser connects to 1.2.3.4 which is a proxy run by your ISP, it accepts your request and checks the path against the IWF list
- If you were requesting /nice, your request is just proxied through to the true origin
- If you were requesting /secret your request is dropped, alarm bells rung etc
A few years back (fuck... 11 years back, it was 2008), this setup led to all UK users being blocked from editing Wikipedia. The reason was they were all originating from an IWF filter box because the IWF had decided that an album cover was "potentially" illegal. It's an old post, but I've got examples of other IWF fuckups on this page.
Things are obviously a bit harder nowadays, because HTTPS adoption has increased significantly. I can only assume they handle it the way they handle things like The Pirate Bay which is to block the entire domain (because they can't provide a valid cert for the domain and therefore can't see the paths being accessed). With things like Torrent sites, they appear to use DPI to check SNI too, in an attempt to try and catch users that aren't using their ISPs DNS servers.
Some ISPs, by the way, intercept UDP packets destined for 8.8.8.8 port 53 (and others) and redirect them to their own DNS servers, so simply configuring to use another DNS server isn't sufficient.