back to article Confusion reigns ahead of comms überdatabase debate

Jacqui Smith will soon begin one of the Home Office's famed consultation exercises on new systems demanded by spy chiefs to snoop on internet communications in the UK. But already, the mangle of powers and regulations around data retention threatens public understanding of what is being suggested. A somewhat confused report …


This topic is closed for new posts.
  1. Mark

    "increased reliance on communications data"

    Aye, but it was always used in 90% of cases.

    "We heard him say 'down the blag at 5 o clock'" is using communications data. And such information is used in most court cases.

    In how many cases was telecommunication data used and its lack would have had the conviction failed with practical 100% certainty?

    Because if you have an arrest at the scene, bringing in a phone conversation with his mates about how he was going to rob the store on thursday isn't going to secure a conviction by its inclusion: the arrest itself is enough.

  2. David

    Anyone Else...

    Read EUDRD as EUTARD? The genericitemtard comment is becoming so common my eyes must've slipped. More coffee!

    OK, I'll get mine now...

  3. Anonymous Coward
    Anonymous Coward

    still don't get it

    I dont' get it, but then I also don't care in the case of ISPs holding onto the data and the filth having to request access and have grounds to access it. That is acceptable.

    However the Intercept database the filth and co want is another matter, they'll just use it for fishing trips, becouse if they're not going to use it for fishing trips then what's wrong with having the data stored at the ISPs and only having access when you have a warrent? Simple, you can't go on fishing trips.

    Anyway seems useless, most of the mail will be spam, the rest will be people sending jokes to their mates or work, it isn't going to help you catch a fly tipper unless you go by the email address "" or a terrorist for the same reason.

    Also what the hell is the point? Most people use online email anyway. Most businesses use their own email servers, and I'm pretty sure any dangerous (as opposed to retarded) terrorist is going to use some form of secure communication.

    So all you'll have is vast databases full of useless s--t or retarded drival... not too unlike my comments. so as said, whatever, ISPs can keep the junk, we can pay for it, and the security services get to jerk off over it and we'll go back to the useless tossers catching other useless tossers whilst bad crap continues to happen.

  4. Will Godfrey Silver badge

    So what's new?

    Our illustrious government still doesn't know it's arse from it's elbow.

  5. Bob

    Intercepted e-mail

    Our intelligence indicates that Mr. Kalid intended to develop a penis of devastating proportions.

  6. Anonymous Coward

    All Your Internet Belong to Us...

    ..demanding that every internet connected host / device in the whole wide world logs every single TCP / UDP connection including date/time, destination/origin, stated business and packet payload (including a decrypted version where non authorised encodings maybe used).

    Meanwhile this data will be aggregated using an adapted version of the software developed to transmit restuarant chain pizza sales back over the dial up link to headoffice.

  7. Martin Silver badge
    Thumb Down

    @still don't get it

    Did you volunteer to pay VAT on all those import DVDs? Pay income tax on your ebay sales? Well we have a list now and will be round to see you.

    Complain about anything to your local council and a list of all the images you looked at on 'readers donkey' will be on it's way to the head of your kids school.

    If the police accidentally shoot you on the tube this can be used to find some link to terrorism. "Well his email went down the same cable as some email from someone who knew someone who had once been to iran"

  8. This post has been deleted by its author

  9. Nomen Publicus
    IT Angle

    Difficult data reduction problems

    Anybody who has ever tried to reconcile email logs from various sources will know it's an almost impossible task. The log formats depend on the software used, time fields are in many formats and sometimes do not include the year (even worse, different sites may or may not use accurate time sources or record their time zone.) Not even host names may be in a useful format for direct comparison.

    It's a mess.

    The government will waste a lot of money obtaining lousy data that can be challenged in many ways by any half competent lawyer.

  10. This post has been deleted by its author

  11. Les Matthew


    "Because if you have an arrest at the scene, bringing in a phone conversation with his mates about how he was going to rob the store on thursday isn't going to secure a conviction by its inclusion: the arrest itself is enough."

    But that might help to prove conspiracy and iirc conspiracy in a charge carries a much heavier penalty.

    Just a thought.

    Personally, this governments path towards total control freakery worries me greatly.

  12. Dave

    @Will Godfrey

    But that doesn't stop them seeing something shiny and demanding to have it "now now now" like a spoilt 2 year old.

  13. Anonymous Coward

    More details required...

    I operate a small datacentre (in comparison to the likes of BT etc) and have just emailed the HO to request clarification of what data they expect to be retained and how they intend to pay for it. We currently keep about 6 months worth of email logs (from, to, ip addresses etc.) for troubleshooting and general inquiries (we have had occasional requests from customers about "did employee x send an email to y in the last few months). If the HO want email subjects logged, that is going to be another issue altogether, as this is not currently logged, and I don't think it is an available option as standard with the software we use.

  14. Anonymous Coward
    Anonymous Coward

    EUDRD plus mechanics

    The scenario I expect is:

    1. She implements EUDRD

    2. She makes her database to receive the data received from ISP under EUDRD via legal requests.

    3. Her database is far too big for the purpose needed. e.g. the £500m SOCA upgrade is 4000 people x £5000 each, leaving £480 million for a big server farm.

    4. ISP's complain about the cost of holding all this data.

    5. She suggests it is stored in her mega database instead, which she just happens to have accidentally commissioned too big.

    6. Perhaps to diffuse Parliament she'll offer a watchdog who'll review their access to that data, or put a form in place to substitute for 'specific instances'. She'll assert that by filling in the form, they are requesting the data in a specific instance, handing control of the database to an outside contractor to make it appear not to be a blanket handing of data to the Home Office.

    7. The form is replaced by verbal approval.

    8. The contractor is sacked and the database move to government HQ.

    So yeh, they're the same thing. She doesn't really need £12 billion for that, the excess SOCA budget is more than enough. She just needs to get it past Parliament (pesky democracy), and she can do that under the guise of implementing EUDRD then shimmy from EUDRD to IMP.

  15. Adam Foxton

    @Still don't get it AC

    Indeed if the gov't are using the idea of "if you don't have anything to hide it won't affect you" shouldn't we be able to ask back "if you know we have something to hide why shouldn't you have to get a Warrant?"?

    The use is more a social-networking thing. Those you email will be noted and a big map of who talks to who (i.e. "there's a lot of foreign-looking fellows talking to some foreign-sounding country over there- let's question them" or- given that the delightful miss smith is involved- "there's a man talking to another man. They're clearly opressing women. Let's lock them up."

    They'll be able to map out the country- and extra-country stuff that passes through ours- based on who's talking to who.

    Also, they'll be able to find out which websites an IP looks at. So they'll be able to make a list of everyone who visits El Reg or Al Jazeera's website or something like that. Though the list will be pretty useless without further information from the ISPs due to the whole dynamic IP thing.

    And it'll also be relatively useless if you use Tor & encryption; as soon as a Tor layer is outside the UK/EU (say, through a Middle Eastern country like where these terrorists are supposed to come from) they'll no longer be able to monitor it- but if a chain is completely within the UK/EU it'll be a hinderance rather than a good defence.

    It's also entirely useless against people talking or setting up wireless links (world record is 237 miles, so I'm pretty sure that a network of people across a city could get a network up and running) or arranging package drops for USB sticks (or even ultra-concealable Micro-SD cards which could remain undiscovered, say, inside a beer mat or inside a display cooker in the local Comet for weeks) or even- shock horror- meeting in the pub and talking about their plans.

    So they'll be able to catch the most dim-witted of terrorists with this proposal, the costs of which will spiral out of control into the billions.

  16. Ashley Stevens


    I don't understand how the expects ISPs to do this? Most ISPs provide only the pipe to the home. My email provider is outside the EU and I use SSL to access it. Many people also use US-based webmail providers like Hotmail and GMail. How can my local ISP that provides my connection be expected to log every email unless it can decrypt SSL and then inspect the packets? Either my understanding is lacking, or (more likely since this is IT) the government's is. Surely all this will achieve is ensuring that everyone uses off-shore email providers? The government seems to be under the misapprehension that email is always provided by this ISP rather than another provider but I suspect this is the except rather than the rule.

  17. Jess

    So what about webmail?

    If someone uses googlemail etc. how would this be recorded, especially if they use https?

    (Also what if they use mailservers other than their ISP's?)

    Will it not mean that ISPs just stop bundling email accounts into the broadband packages?

  18. Anonymous Coward
    Anonymous Coward

    Re: great

    If you're using the server for email then at least one form of connection is not encrypted...

  19. Anonymous Coward
    Thumb Up

    @AC More details required...

    And please tell us if you get a (comprehensible) reply from the HO fucktards.... Actually, tell us anyway whatever they reply (though I doubt it will be anything near comprehensible).

  20. amanfromMars Silver badge

    A Helping Hand

    "Our illustrious government still doesn't know it's arse from it's elbow." ..... By Will Godfrey Posted Friday 9th January 2009 14:43 GMT

    Such is the nature of Parliamentary Democracy with no Leadership, Will. But that is not to say that the Private Sector cannot/will not Supply them with an Access Portal to whatever it is they want. Surely that is ever the case in these special cases, otherwise one would have to conclude that , when one has to set a thief to catch a thief, for Government to design a System for catching thieves or whatever, they would be thinking like them. And who would want such thinkers tinkering in Governments. That would be a recipe for Disaster/Meltdown/Credit Crises etc etc.

    Where can one scan/download the Desired Specifications of IMPerative?

  21. Anonymous Coward


    "The propaganda arm of Brown's regime, controlling information: news, entertainment, education, and the fine arts. Jacqui Smith works for the Records Department (RecDep) of Minitrue, "rectifying" historical records and newspaper articles to make them conform to Big Brother's most recent pronouncements, thus making everything that the Party says 'true'."

    Wow. I changed 2 words there. And it fits.

    Mine's the one with tinned foot, tin foil hat and AK-47 dans le poche.

  22. Tony


    "it's been used "as important evidence in 95 per cent of serious crime cases" since 2004"

    The contents of emails are not admissable as evidence in UK courts except in very specific circumstances - data on who sent and received the email is allowed / IP addresses as well are also acceptable. (I'm currently studying a course involving computing and the law and had to write a short essay on the topic of specifically what cases the contents of emails can be used as evidence)

    I now tend not to believe anything Frau ReichsKommandant Smith has to say, as it tends to be based more on her daydreams than on fact.

  23. Steven Jones

    Encryption and so on

    Of course it is perfectly possible for anybody to mask what they are up to by using encryption, anonymising proxies, out-of-country email servers and so on. However, what you can't hide is that you've initiated an IP session to one of these entities. I've no doubt that the government wil rely on the vast majority of people not bothering about this and that the patterns of access of those that do would provide for intelligence information, although it might not provide courtroom evidence. It would mean that if a person was suspected then a much more detailed investigation could take place. I have no doubt that there are already powers to do more detailed tracing of an individual subject to the appropriate legal authority. Also, if a PC was seized which had been used for such things then it's very likely evidence could be found (even an Internet cafe).

    What the EU legistlation does not allow for is data trawling requests - that is simply ploughing through huge amounts of data looking for suspicious behaviour. For that I primary legislation would be required. I've no doubt that the intelligence community would simply love a huge database listing every IP access be everybody in the country (and to everybody in the country). However, the data volumes to do this are mind boggling. They are bigger, by orders of magnitude, than the data for phone calls.

    As far as the EU legislation not having any impact on the data that the large ISPs keep, I rather doubt that's true. I suspect that most large ISPs don't keep information about every IP address accessed by every individual and relevant time stamps for periods of a year. They simply don't need that much information for their purposes, except on an exceptional basis, and it's not needed for billing (where this is relevant, then summarised patterns are likely to be enough). I've no doubt they keep summarised activity records, IPs allocated and so on for a lengthy period, but itemised data, even without the content data about each connection made would be simply enormous..

  24. Anonymous Coward

    Just use..

    Swiss VPN, job done.

    The new wacky jacky plans will only catch idiots and chavs who mug corner shops. Everyone else will get round it.

  25. Will Godfrey Silver badge

    Just another thought

    There used to be a competition to see how far you could get on just 1 watt of CW radiated power (morse code). I think they did actually get from England to Australia once. If I was part of SMERCH then I think that would be the comms method I'd be investigating not the interbabble.

  26. Anonymous Coward
    Anonymous Coward

    @@still don't get it no. 1

    Not too up on reading are we.

    That's why the filth shouldn't have an uberdatabase themselves. Becouse they'll go fishing. But there is no problem with the ISPs storing this data becouse the filth can't go fishing without a warrant.

    I don't mind information being stored, however it should never be centralised and most certainly not under the control of the people that most want it, the fuzz, politicos and "security" forces like any other large kid can't help but dip their hands in the cookie jar.

    If the police suspect Mr Smith of being a terrorist or a has commited a crime and they have evidence to back up their claim then i have no qualm with them obtaining a warrant to gain access to Mr Smiths communications, banking, employment, or any other information that may aid them in their enquiries, if their enquiries turn out to be baseless, no harm, and no storming of a house, shooting the guy, and faking child porn on his PC just so the filth don't look quite so racist and inefficent as they are. (Evidence for warrant should be a bit better then we saw him at the local church one night and he looked funny.)

    However I don't believe that the filth/politicos/secuirity forces should be able to go "search for everyone that who matches these user habits in t3h database, or complies to this algorithm" it'll almost never catch a smart criminal and just throw up false positives, and probably has the same chance of getting a criminal as arresting everyone with the surname Khan or Smith or Jones, they could just wander into asda on a sunday afternoon and ship everybody to the cells so they can be processed, that would probably yeild just as good a result as fishing trips in their uberbase.

    If you were a group of terrorist, just create a terrorist network, there's loads of stuff you could use to create a network that would make tracing you almost impossible.

  27. Anonymous Coward
    Anonymous Coward

    Makes it harder ( and I'm not talking viagra!)

    By adopting this kind of strategy, Smith is actually providing an incentive for the terrorists to get round the measures, so it will probably be harder to identify what the bad guys are actually up to.

    But don't dare mention this kind of argument to her, she's not likely to understand it.


  28. Anonymous Coward
    Anonymous Coward

    @AC 15:50

    > If you're using the server for email then at least one form of connection is not encrypted...

    That's frequently not true.

    Client connections can be over an encrypted link (e.g. https webmail, which is what I use), and the MTA can use TLS (which seems to be becoming the norm, except for connections to/from MS Exchange)

    So two identically-configured[1] web/mail servers gives you end-to-end encrypted messaging. With out-of-band key exchange, you can even get the MTAs to verify each other to prevent MITM attacks.

    [1] This particular configuration is also the default configuration for the distro I use.

  29. Will

    I hope I've just gone and read the right thing, as that was really boring

    Ok, sharing confusion with other commenters on how this should work, I've gone and found what I think is the directive, and done my best to understand it.

    Points I've noted:


    Number 13 of the "whereas" bit at the start states: "In particular, as regards the retention of data

    relating to Internet e-mail and Internet telephony, the obligation to retain data may apply only in respect of data from the providers’ or the network providers’ own services."

    which I assume means that if you make sure you're using different email providers and voip providers to your ISP you're not being tracked. Which in turn suggests that this is almost completely useless for intelligence purposes.


    This isn't just about emails. It also covers voip calls, sms messages, etc.

    3) This only appears to relate to companies responsible for the transport of the data (ISPs), so if you happen to run a data centre I think you're ok (but don't take my word for it).

    Am I on the wrong track, or have the UK implemented a more detailed law?

  30. The Fuzzy Wotnot
    Thumb Up

    From what I gather...

    They are only going to keep time, date and subject? Well I hope all you naughty crims and terrorists are going to remember to put "Fred, we must blow up XYZ on the 5th" and "John, we must nick stuff from building ABC on the 9th" in the subject lines or we won't have anything to come after you with!

  31. Dave

    Home Email Appliance

    I can see a market for a small box, possibly a modified NAS type box with an IMAP server and TLS-enabled Sendmail on it and a USB memory stick for storage. Set it up along with your own domain and MX records pointing at you, and then you've got your own solution. Incoming email comes straight to you encrypted and is stored on the memory stick, you can read it via the IMAP server from your home machine(s) and anything you send can go straight to your mate's box and not touch an ISP server at all. It shouldn't be too hard to do, I've got most of the bits, I just need a simple web interface to help Average Joe configure it for his system.

    Chances are it's already on Sourceforge somewhere and I just haven't seen it.

  32. Graham Marsden
    Thumb Down

    "famed consultation exercises"...?

    I think the word you were looking for is "infamous"!

    And from that BBC article:

    "The Home Office said the data was a vital tool for investigation and intelligence gathering."

    I think they meant "a vital tool for fishing expeditions and data trawling and the hope that, after the event, we can find where the criminals came from".

  33. Andy ORourke
    Black Helicopters

    Moving Targets

    Ok, lets assume the government get the Uber database up and running, lets also assume that those tech freaks (who don’t have anything to hide but simply dislike the idea of being snooped on) decide to start using encryption, SSL and other techniques to try to cover what they are doing (no matter how legitimate, you know, downloading your latest Linux ISO ;-) etc)

    Come on now tech guys, think about this, the ENTIRE IP traffic for an ENTIRE country is being stored & monitored. Just WHO is going to be looking ayt it? That's right, NO ONE but think about what would raise the alarms bells..........

    Spook 1: oh look, the system just flagged up that IP address just initiated a secure session, encrypted as well over an SSL link via an overseas proxy, lets be having a closer look at that then.

    The vast majority of people in the country won’t know about this.

    Of those that do know about it the vast majority won’t care about it.

    Those that do care and have the will / knowledge / desire to do all they can to obfuscate their internet habits, you just became the potential target.

    Helicopters because, well, you know

  34. Skyraker

    Has anyone figured out how much....

    .... data this would generate?

    I'm too stupid, but can someone have a go?

  35. mh.

    Building a bigger haystack

    I hope the intelligence services like reading terabytes of spam every day, especially when it usually gets sent from fake email addresses. Most spam is sent via SMTP servers on botnets where the PC owner often isn't aware that their machine is pwned. Will there be requests for information sent via these machines? Alternatively, what if the baddies decide to use botnets for their own purposes? I'm still not convinced that swamping GCHQ with garbage will improve national security.

  36. Anonymous Coward
    Anonymous Coward

    Information Overload

    I am not technically savvy, but from what I know about encryption, I tend to agree it may make an internet user more of a target for surveillance. Researchers have proved the Tor network is vunerable to certain types of surveillance & it is believed by some it is actively spied on (it has been used to try to hide some illegal activities). Plus I'm a bit skeptical about using what was originally a US Navy-funded project. Surely the answer lies in safety in numbers? Use of the internet is growing exponential. One headline I glimpsed just today suggests Blu Ray is already on the way out to be replaced by downloading movies online. The more information users generate the more information there is to analyse. And the more random the more difficult it would be to build profiles. I use the Firefox add-on SquigglesR as it generates false search queries & clicks on websites. That should provide some cover.

  37. Anonymous Coward
    Anonymous Coward

    @By Jess

    last time I checked, gmail via https, actually part of it goes via http, which contains your email account name in it.

  38. Anonymous Coward
    Dead Vulture


    I can't believe your government actually believes it's feasible to warehouse every packet sent across the UK. Obviously, the costs WAY outweigh the benefits.

    1. It's a HUGE invasion of privacy. As if the spooks being able to sift through this database weren't bad enough, could you imagine the repercussions if it fell into the hands of a criminal or, worse yet, a hostile intelligence agency? I can't believe there hasn't been a public shitstorm at mere suggestion of such a database.

    2. It's technically infeasible. While I can't seem to find any statistics on how much data crosses the wire daily in the UK, I imagine its in the hundreds of petabytes at least. The cost of warehousing all that data would be utterly ridiculous and will only increase with time.

    3. It'd be impossible to yield useful data from the database. Sure, you might catch a one or two "sympathizers" and the like who may not be motivated enough to take proper precautions, but for the most part I think the knowledge of this program would motivate true terrorists, supposing they even use the internet for planning, to use encryption, stenography, proxies, tor, etc.

    4. It'll have a chilling effect on internet-based businesses in the UK. I know *I* sure as Hell will be more reluctant to use internet-based services in the UK with the knowledge that all my data are belong to the government.

    5. Why is there this double-standard for electronic communication? Would UK citizens find it acceptable if all their face-to-face conversations were recorded and databased for the broader public "good?"

    CCTV cameras, facial recognition, DNA databases, national ID cards, mass surveillance... Liberty is dead in the UK. RIP

  39. blue
    Black Helicopters

    It doesn't matter that it's imperfect

    They don't have to get it right at the off set - this law is about establishing the legality of monitoring the behaviour of all citizens, all of the time. It is about establishing, in law, the right of the state to put all of us under surveillance, to note what we read and who we communicate with.

    It doesn't matter that this law doesn't take into account webmail. It doesn't matter that IM is overlooked or that VPNs or TOR can get round this, it doesn't matter that you can avoid this surveillance by any number of different ways. Once this law passes, and remains in place, un-challenged, it will effect a change in the relationship between citizen and state and will say that ordinary citizens are subject to 24 hour surveillance of everyone they communicate with and everything they read (online) by the state.

    Loopholes will be closed, protocols will be added. VPN use will be subject to regulation. The Surveillance State will have been established and will then, as little more than a matter of routine, go about perfecting its means of operation, tightening its grasp, under an already established 'mandate' of total surveillance.

    The literature we read online will be monitored. Our political affiliations will be seen and recorded. Our religious bent will be noted. Our sexual pecadillos will be subject to government scrutiny.

    This has no place in a democracy. It is the action of a police state. It is proof that we live in one.

    All citizens, suspects. All citizens under surveillance. All of the time.

  40. Mark

    @Les Matthew

    And in how many cases were that needed? If your terrorist is in jail, they can't blow anything up.

    Doesn't matter if they're in for five years or ten.

  41. Anonymous Coward

    @ Blue

    I agree completely with that comment - not quite ready to have his or her babies though.

    In '1984' the only way for Big Brother to have our minds and souls open to them was to torture us almost to death - these days all they have to do is read our comments and blogs on the internet. Then marry up our internet activities with a picture, name, number, biometric data and...

    ...they have us mind body and soul 24 hours a day for the forseeable future.

    Even 'laid back' Australia is getting in on the act and Amsterdam have been closing weed cafes.

    What gets me is the probable end result of a police state - the destruction of history, cultures, unguided creativity, starvation, delapidation, poverty. What the hell is the point?

  42. Anonymous Coward


    wouldn't it be great if the IT people they required to create this useless waste of money, makes you wonder how much goes directly into higher ups pocket, just said, "No, will not build that for you."

    I culdn't afford to say NO, but wouldn't that be great.

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2019