Why are the ISPs making such a fuss?
The IPB is calling for ISPs to retain data about the sites we visit and hold it for a year. This is a small subset of the data they already routinely collect and store for much, much longer (and who knows what use the ISPs make of that data).
ISPs operate what amount to proxy servers. The are well known bits of software used in most companies to make sure the company knows what staff are looking at and are not giving away the crown jewels. Just like web servers these routinely log the IP address and page request of user requests for a benign reason: to be able to trace technical issues if they arise. These logs can be text files or database but whatever form is used, the logs will be scooped up and added to the long backup as part of the standard disaster recovery process.
The IPB doesn't ask for all the detail already, and routinely, stored. Instead it asks for the IP address. Put this into context. When you visit a site and your browser renders a page it may in fact be the result of tens or hundreds of requests as the browser separately retrieves images and other components needed to be able to render the page. All this detail is stored by the ISP *for every component*. You may remember that exploiting this information was at the heart of BT's 'Phorm' project a few years ago. By comparison, the IPB requires just a tiny fraction of this information is retained.
The information required to be stored is my IP address and the IP address of the web site. An IP (version 4) address stored as text is 15 bytes so that's 30 bytes per visit. Say I visit 100 sites a day. That's 3K per day or 1MB per year or substantially less than size of a smart phone picture to hold my site visit information for a whole year. Lets suppose there are 40 million UK subscribers. That's a storage requirement of 40TB per year. The cost of a 4TB disk is now less than £100 so that's £1000 to cover the storage requirements for the entire country. Now I know the ISPs will use much more robust storage so lets say the cost is, really, 100x this amount £100K. That's across *all* ISPs. Its a trivial amount.
Now the ISPs will know this. So the question is: why are they making such a fuss? Maybe there's concern that if they capitulated too easily it may provoke someone to ask why are they not making a fuss because it may expose the extent of the information about us the ISPs already hold and use in ways about which we have no idea.
The US government is pursuing a case against Microsoft requiring they disclose email information held on a server in Ireland. A US Judge has agreed and Microsoft's appeal has been heard and is being considered now. The basis of the US government demand is that data held by any US company anywhere in the world can be requested by the US government - remember this has been approved by a US Judge. Virgin Media is owned by the US company Liberty Global. If Microsoft loses the appeal, it seems that the US government will be able to access all the browsing habits - not just the IP address - of all all Virgin Media customers because VM is, in fact, controlled by a US company.