back to article Spy chiefs plot £12bn IT spree for comms überdatabase

Billions of pounds of public money will soon be up for grabs for private IT contractors ready to serve the Interception Modernisation Programme - UK spy chiefs' plan to store details of every call, email, text and web browsing session. And The Register has learned that one plugged-in firm is better poised than most to profit …

COMMENTS

This topic is closed for new posts.
  1. b
    Flame

    Coming soon...

    You will be billed a grand to have your anus-cam fitted.

    Only a terrorist or a pedalo could possibly object.

  2. Anonymous Coward
    Anonymous Coward

    Lets get real here

    Look, here's a nice documentary from a few years back explaining what's happening now:

    http://www.youtube.com/watch?v=Q3uvzcY2Xug

    Notice the Iranian oil executive and the chairman of Cheney's Energy Task Force, explaining that Natural gas has peaked and oil is expected to peak in 2005-2007, (which is backed by the numbers now) and that essentially our core food supply is at risk and suburbs are unsustainable...

    So right now we face some incredible challenges that will require all of our wealth, and quite frankly obsessive control of the public by an incompetent twat who should have been sacked a long time ago, is not one of them.

    So let's get real hear and not waste 12 billion on shoring up an unpopular government and spend it instead on energy and food.

  3. N

    Bargain...

    Data Tsar, over budget, Knighthoods a plenty & complete utter failure ahead...

  4. Mo

    Not being funny

    But I'm not sure we've got £12bn to spare, let alone what it'll *actually* cost.

  5. Anonymous Coward
    Stop

    so who does the maths here?

    We have so many projects going including olympics, this the war etc etc and all is funded by the tax payer without any consent from the taxpayer to add insult to injury there is also a credit crunch with banks failing and again the tax payer asked to pay for it ?

    How much tax can you gain out of ones wages ?

    Is it time we all stopped working and went on the good old DHSS ?

  6. Cris Page
    Flame

    Speechless... or we soon will be

    We cant equip or house our troops properly, we want to bully disabled people back to work, Fuel is taxed at a level that can only be discribed as outright robbery and yet we can find acres of cash for the multitude of databases they want to set up. Govt must be really scared of the populace, if only we were not so spineless as a nation we might have already ousted Gormless Brown and his Feu Liebore rabble.

    Whatever happened to the right to privacy as mentione in the HRA? Oh I forgot as with most things - this Givt only accepts the bits it wants... and ignores that which goes against its control freak nature.

  7. Anonymous Coward
    Anonymous Coward

    And this requires new dev work because?

    We did something similar in scale and technology some 8+ years ago and the project is still running. In fact we won an award for being under budget and well within timescales.

  8. Tom Chiverton

    "Trust trumps everything in this field"

    And *anyone* trusts a government with this soft of thing ? Geez.

    Never mind what the UN convention on human rights (such as privacy) will have to say about it.

    If any company wires this into their services, I'll just do with out them. I can always send all my IP traffic via SSH to another country, and get a nice IP phone to boot.

    Incidentally, the ID card project is at least 10 billion quid, so could well go past the NHS and IMP once the overspend occurs.

  9. Anonymous Coward
    IT Angle

    Old news

    Surely I've read all this before?

    I've known about this project for a hell of a long time certainly.

    So what's new?

    Just use lots and lots of proxies or Tor or if you have your own business then use a VPN.

  10. Anonymous Coward
    Black Helicopters

    So easy to game this database

    Just need a plugin for your email client of choice that adds a string of potential target words and phrases, like "bomb plot", assasination, etc to every single non-signifiant email you send, i.e. the ones with suggested locations for the Friday night drink-a-thon, or the one about that latest hillarious viral . If enough people sign up all the false positives will bork it completely.

    Then use pgp for anything that is significant.

  11. Strappy

    "Game this database"?

    Some commenters appears to have missed the point of this project. It's not going to scan content (there's already plenty of systems that can do that), it's going to map the connections between people. Only the most naive terrorist would send an email containing such obvious trigger words as "bomb" and "assassination" and those emails would doubtless be flagged and filed but I doubt serious action would take place.

    If the Security Service suspects that a potential terrorist cell is communicating with other cells, however, they will be able to track communications through IMP to locate the others even if those comms are fully encrypted. Sure, there are wider implications for personal privacy but I don't recall seeing mass protests about the number of CCTV cameras that watch your every movement.

    It's exactly the same type of data gathering as carried out by supermarkets through loyalty cards or Phorm targetted advertising that's been reported by El Reg recently. Knowing how people move about, communicate or even idly surf is now valuable data for mining.

    To bastardise the Marshall McLuhan quote, the medium has become the message.

  12. Anonymous Coward
    Joke

    gaming the system

    I feel a new website idea coming on...

    Spookvortex.net

    Wherein users can meet Others possessive of a Disdain for Wholesale Spying on the Populace 42 share Email Addresses with the Express Purpose of exchanging NSA-Line-Bombed Conversations, Snippets of Suspicious Non-English Script and PGP-Encrypted chunks of Pure Random Nonsense in order to Gum Up the Works of their Fiendish Yet Deadly Parlour Game of Six Degrees of Suspectibility.

  13. michael

    hold on a sec

    "UK spy chiefs' plan to store details of every call, email, text and web browsing session."

    so that is who phorms chief customar will be

  14. Chris Cheale

    *sighs*

    Well that explains why the Met didn't push for prosecutions on BT/Phorm then... El Gov actually wants it to succeed so that they can have someone else do all the dirty work and they can then just nab all that lovely data.

    Arsehats.

  15. Anonymous Coward
    Thumb Up

    Never gonna work!

    You know IT in the UK is crap and IT under government control is the very bottom of the crap pile. EDS or some such bunch of twats will get it, cock it up, run over budget by about 5 times. Finally it will emerge a half -backed useless failure that only fulfills about 10% of it's original goals due to "all the cooks spoiling the broth" and the fat pigs at the IT money free-for-all trough! Me? I'm gonna brush up the CV and get me a contract on this little baby, license to print money ain't in it!!!

  16. Mungo
    Linux

    Its already started....

    .....this sounds like the back-end to an out-of-band, deep packet inspection personal data harvester from hell but they dont have one. Funny to announce such a huge backend end database without having a licensed technology to collect personal information and browsing habits without our consent or knowledge.

    Wait a minute though, isn't that exactly what BT is currently testing and the Government is desperately trying to convince the EU et al is not illegal even though they would have to rewrite several existing laws and a couple of EU ones to make it legal. Hmmmm, now things begin to make more sense.

    There's bound to be an opt-out screen though.....isnt there?

    Tux because putting our OS differences aside......we are all screwed equally.

  17. Anonymous Coward
    Flame

    60,776,238 people.

    Which means probably nearer 30million taxpayers. So this will cost me, you and all the people around you (assuming you're at work) £400. Assuming it doesn't go overbudget. Add on another £380 or so for the ID card programme and probably about a grand for the olympics. Now even for someone on 100k a year that's over 1% of their salary gone on useless, doomed-to-fail, non-profitable (in any way- not just money, but increase in safety etc) projects.

    You know what, I'd be happier if a private company was doing this. Yes, they'd probably be evil and moneygrabbing and so on- but so's this government. At least they'd be doing it for SOMEONE'S benefit (shareholders etc) rather than just "because we can".

    To any of you Che Guevara (look him up, nice guy- killed innocents and is a hero for it...) worshipping anti-capitalist fools out there as well as those who believe that labour is a "working man's party" and even if they were true that this somehow makes the useless bunch of Socialists suitable to run a country, this is the world you've ushered in. Or rather are in the process of ushering in.

    Christ, we had more freedom under the monarchistic theocracy we used to have.

    And I'd just like to help Phil out with his idea. A selection of random words for ya: "PARLIAMENT" "BROWN" "OFF" "FUCK" "AND". Oh, and "BOMB".

  18. paulc
    Black Helicopters

    better still...

    use pgp for everything... and campaign for website operators to switch to https for normal browsing as well...

  19. Neil Barnes Silver badge
    Black Helicopters

    Twelve billion...

    That'll be two percent of our average tax payment over a year, then, just to look at my emails?

    When do we roll out the completely end-to-end encrypted internet?

    <echelon><echelon><echelon>

  20. Davey Bee

    State monitoring

    This is desperately serious - though many people may not appreciate it. Years ago we despised regimes which spied routinely on their citizens; which employed secret informers in every town; and which would not allow their citizens the means of communication or publication (typewriters were banned in Communist Romania, I believe).

    It really does not matter what these regimes might have called themselves – right-wing, left-wing, whatever – the point was that totalitarian control over citizens’ lives was viewed in this country with outright horror.

    But left wingers in particular have always yearned for greater state control; this is their natural and probably well-meaning aspiration. And now we see the logical results: no private communication is to be allowed. The state is now all, and the left have to see that this is a monster they have encouraged… and despite their gut feelings, they must come out against it.

    What is to happen to me if all my outgoing emails are now sent with an encrypted attachment containing my message? Will the state consider that to be suspicious in itself? What about meetings in publlc parks with a few friends? Will the police want to know what that’s all about? If I criticise a politician, will I be reminded by some civil servant that I was recorded the other week as having searched for “Milfs in leather” and that it might embarrass me if that little secret were to get out?

    Things are getting bad, and if we’re not careful we will be right into a situation which Brezhnev, Hoenecker, Ceausescu etc. would have envied. Let’s not allow it.

  21. spam
    Thumb Down

    Politicians!

    Funny, no problem pissing away 12 billion quid soothing the ego of politicians who think we will thank them for the protection against non-existent threats this system will offer.

    Elsewhere today I read europe has managed to find (in comparison) a few peanuts (63 million euro) to fritter away on fusion reactor research.

    Hmm locking up a few raghead camel jockeys that visit the wrong websites or striving for a new clean plentiful power source for the world - which would you like your taxes spent on?

  22. Anonymous Coward
    Anonymous Coward

    PGP ??

    That would be the encryption that you legally have to hand over the keys to or prove you don't have them then, yes ?

  23. Ash
    Pirate

    No more World Wide Web

    Viva la DarkNet.

    El Reg should start posting these spook stories on Freenet.

  24. Anonymous Coward
    Happy

    Just a thought...

    I'm sure I could write a little (OpenSource) utility to send-out thousands of garbage URL requests over t'internet while I wasn't using it, if a few thousand others did the same...

  25. Anonymous Coward
    Coat

    Looks like a business opportunity

    Well I suppose even for those who are not given the billions to implement this, it could be good business say an offshore service to create an anonymous forwarding host, you vpn in and it then send your mail etc. Only problem is it would probably be run by the Russian mafia.

    Or maybe something simpler, say a dead letter box arrangement, but it could be better than a physical one, having multiple entry points and no logical connection between them.

    I'll get me coat, it the one with the big book of passwords,encryption keys and a mirror to spot black helicopters, in the pocket

  26. Anonymous Coward
    Anonymous Coward

    What do you mean, the most costly project to date?

    There was a grenade attack on the MI6 building at some point.

    The grenade hit a window somewhere half up the building, exploded AND THE PLACE DIDN'T HAVE A SCRATCH (other than a bit of blackened paint). In other words, this place is top to bottom hardened.

    That surely will have costed more than 12bn, but that cost is conveniently hidden from public view..

  27. Anonymous Coward
    Coat

    But....

    "Only the most naive terrorist would send an email containing such obvious trigger words as "bomb" and "assassination" and those emails would doubtless be flagged and filed but I doubt serious action would take place."

    Oh really? Well that means that when they do flag and file all the emails containing the word bomb, the rest of us sheep will be expected to proved that we didn't have anarchistic tendencies when we used the word bomb, and to explain why we chose those particular words, and probably taxed, sorry fined for doing so. Ridiculous. As you say yourself, it will not incovenience terrorists at all. (I, on the other hand would probably be hauled in to explain why I used the word terrorist too, after all, if I have nothing to hide or fear, why would I be using subversive words like terrorist?)

    WHY should I have to explain to anyone what words I use? WHY should I have to be careful what I say in a free country? Oh wait...

    And it isn't just the connections, it is every communication, every SSL stream, and 15 years ago we thought it impossible to decrypt what now seem, with current hardware and sophisticated software, like simple cyphers. In 15 years time, it could be trivial to decrypt all those lovely SSL sessions we have stored...

    Mine's the one with the little amateur radio antenna on it.

  28. Brian
    Stop

    Heil Braun!

    This Neues Arbeit lunacy is going too far. There can be no justification for spending 200 quid per head on a system that is likely to be rampantly misused to track dissent - if anyone thinks that this will be restricted to tracking 'terrorists', they must be either naive or stupid.

    Anti-war protesters? Check.

    Fuel protestors? Check.

    'Extreme' political parties? Check.

    Want to anonymously highlight council corruption? Check.

    I have no doubt that if this system is ever successfully implemented, RIPA will allow the number of people with access to the data to expand as far as the council dog wardens - your associations and browsing habits will be fair game for any petty bureaucrat with a little hitler complex.

    Bah humbug!

  29. Ash
    Flame

    Control control control

    Too open to abuse, too open to cockups, too little benefit, too much cost. This plan is fkin shite tbh, spend that money more effectively elsewhere and stop trying to become the Stasi (or worse).

  30. Matthew Ellen
    Stop

    They're not reading, they're watching.

    I know someone's already said it, but just to remind anyone who has forgotten between there and here: they're not storing the messages, just the source and destination of the messages.

    I'd ask any IT contractors to decline opportunities to work on IMP, but I guess we all have a price, and £12bn is a big flavoursome pie to want a bite of.

    Won't someone think of the children?

  31. Nomen Publicus
    Black Helicopters

    Lots of dead Brazilians?

    Yesterday the police claimed that they did nothing wrong when they shot a man seven times. Apparently he did a lot of suspicious things like get up, leave the flat, catch a bus, change his mind and finally decide to travel on the underground.

    So they shot him.

    Why would anybody think that this crazy scheme would be any better at identifying the bad guys?

  32. EvilGav

    @strappy

    The difference between this/Phorm and store loyalty cards, is that with the latter I have to actively partake in them.

    And isn't the ID card debacle estimated to actually cost £25B ??

    It's a good job we aren't in a situation where fiscal responsibility is necessary . . .

  33. Scott
    Thumb Down

    Point to..

    Could someone point me to a NuLabour Left wing joke of an IT project that actually works? I surpose you could say ID cards if they ever get of the ground but as there running many times over budget and years behind schedule can you say this project is successful? i'm guessing nope.

    Hey Browny heres a tip you put us in this situation where millions of people are below the poverty line by being chancellor for years why not stop wasting money on crap like this and help the people that have to choose between heating and food?

    Or if you really want to burn all our taxes rather than pay the mortage back as i think i'm right in saying where one of the most debt ridden countrys in the world? why not just start a massive fire outside number 10 and burn the billions in a big pile i'd pay to watch that?

  34. Matt Bryant Silver badge
    Pirate

    This is sorting, not collection

    This whole database project is about sorting, storing and retrieving data. The Government already has the information in disparate sources, they just want to make it easier for spooks to match up all the bits of the jigsaw.

    PGP encryption is a big "come look at me" sticker. However, it's just a sticker - they can eliminate you as a subject of interest pretty quickly. PGP + no other points of interest = paranoid, ignore. But PGP + visits to jihadi websites + emailing Achmed in Pakistan = intercept and mark for further attention. And if they want to decrypt they can order you to surrender the keys or face preosecution. Not smart.

    And finally, for all idiots that want to spoof the system because they are irate at the expense, you should stop and realise spoofing will just mean more data recorded, stored and evaluated, which means more processing and therefore more cost. Grow up.

  35. Wonderkid
    Stop

    Why not invest these billions...

    ...in improving life for people here in the UK. If it's a nice place to live, less people will want to blow it up. A large investment in public information displays for example. Have you ever stood at rail, but or tube stops and wondered why in the 21st century you're faced with useless, obsolete or missing route and/or timetable information displays - if any at all.

  36. Dave
    Black Helicopters

    @Davey Bee

    I totally agree with you - the worst thing is that other countries believe the same thing and this idiotic belief in technology is becoming even more rampant. Only look at the USA for further examples of lunacy and the whole idea of "...if you have nothing to hide, you have nothing to fear..." is just plain wrong.

    The major problem we have though is that the majority are so poorly educated on this type of thing. They read "The Sun" or the "Daily Mail" or a.n.other source of (dis)information and that tells them what to think. Years of being fed garbage has left a populace that no longer cares and is sleep walking into George Orwells vision of the future.

    It is a very scary time, and I just hope that the people wake up and get together - otherwise it will be too late for any action.

  37. Luther Blissett

    Fails sanity test

    If it is not monitoring content, then it is only collecting origin and termination addresses (plus other bits of data, e.g. time, geography). But systems are already doing this, and the records can be accessed, e.g. for drivers involved in car accidents, suspects in detention, etc. Presumably it does not cost $12bn to siphon this data somewhere for archiving, e,g, to keep RIPA-loving local council jobsworths happy for years to come.

    Could it be that the new system is oriented not to data storage but data processing? Is the idea to feed it all into the Mother of All Neural Networks (take it easy now, amfM ;) to look for patterns? If so, it can only be for use as a real time or virtually-real time process, since if you have nabbed your suspect you have lots of time to dig out the communications records (even after celebrating), and it doesn't cost £12bn to dig an armoured hole in the ground, even if you surround it with a shark tank and the sharks wear frikkin lasers. So we are talking massively parallel processing systems.

    A MoANN may be feasible, but is it credible? Latest tittle-tattle from Big Media suggests our spookies now think of islamic terrorists as non-partisan unaffiliated individuals who may or may not be islamic, i.e. anybody and everybody. Which is tantamount to admitting there is no such thing as a useful profile. So it is hard to believe that this thing will do anything other than at best profile a lot of false positives - which is no real help at all, although regularly framing a PSB (poor sad bastard) for high profile crimes does wonders for public morale, or so it is assumed, since it goes on and on.

    If a MoANN makes little sense, then only non-deterministic data mining processing makes any sense at all. But if that is the case the sheer quantity of data, and the requirement to avoid the processing being permanently i/o bound so it can process synchronously with input, indicates some kind of insane hardware monster. To achieve what exactly - that remains unclear. Perhaps it is not intended to achieve a lot more than cause the transfer of interest from taxpayers to banks. It appears to confirm what we all know - Government does not understand the first thing about IT.

    Debits and credits. Will it save lives? Answer unknown. Terrorism is not a major cause of death in the UK. Will it cost lives? Undoubtedly. There will deaths resulting merely from the diversion of electricity from the cold poor this and coming winters to satiate this monster and its creators. The opportunity cost of £12bn not spent on the NHS or alleviating the miseries of the lower classes can also be counted in fatalities that could be avoided (MRSA, ambulance service cuts, hospital reorganisation, NICE's drugs gate-keeping rationing, post-code lottery treatment, and so on). One must estimate in the 000s.

    The clinically insane do not always gibber or foam at the mouth. The ones I refer to are the ones who don't have to use NHS dentists.

  38. Pid
    Go

    Can I stop paying for backup services then?

    If big bro is storing all my emails, phone calls & texts, all I have to do is email (another personal account) my important files and they're instantly backed up.

    The DPA says I can pay £10 for all the data an organisation stores about me, so all we need to do is lobby for rapid access to one's own stored data, right?

  39. Steen Hive
    Stop

    @AC

    " this is the world you've ushered in. Or rather are in the process of ushering in."

    Fuck off. This is modern capitalism to the core - feudal levels of oppression complete with robber barons and a damned sight more efficiency. What a knob.

  40. Ash

    @MI6 grenade attack

    I suspect basic physics (conservation of energy) is enough to explain why the building sustained no damage.

    I have a friend who worked at Cadbury's briefly, who told me about their sugar storage silos. Apparently, so he says, sugar dust is explosive, and you get a lot of sugar dust when filling th silos. Should some bright spark (sorry...) be smoking when he's filling the silos, they've built these warehouses with two brick walls, two thin walls, and a light roof. The (proven) theory is that if there ever IS an explosion, all of the energy will collapse the thin walls, the roof will fall and cause little or no damage, and the building can be put back in working order quickly and cheaply.

    The purpose of that was to state that all the energy went away from teh building, as the easiest route for it to escape.

  41. Anonymous Coward
    Flame

    Gentlemen, start your cryptographic engines

    Encrypting everything is the only way. Sure they can subpoena your crypto keys but at least you then know you are being spied on.

    Bit concerned about layer 7 interception (a la Phorm) however. Anyone want to bet that Phorm, with ex-BT crony on the board and who are obviously lining the pockets of someone senior at the Home Office is putting a bid in for this?

  42. Eduard Coli

    A head

    Well imagine that, North Korea is not the fascist totalitarian state everyone thought it was but was rather instead ahead of the curve.

  43. The Other Steve
    Flame

    @Strappy

    "Some commenters appears to have missed the point of this project. "

    yes, yes they do, you prime amongst them.

    "It's not going to scan content"

    We're told. But you've missed yet another pint, although you're by no means alone in that, the name of the project is the "interception _MODERNISATION_ program", which rather suggests an upgrade of present capabilities. If you think that present interception capabilities don't extend to content monitoring than you're living in la la land.

    "(there's already plenty of systems that can do that)"

    Name some. I can think of one. I'd be interested to hear of the others.

    "it's going to map the connections between people. Only the most naive terrorist would send an email containing such obvious trigger words as "bomb" and "assassination" and those emails would doubtless be flagged and filed..."

    You have just contradicted yourself, postulating here that content will indeed be scanned.

    "but I doubt serious action would take place."

    Why ? There have already been high profile cases where people were detained and charged based on such things, or don't you watch the news at all ?

    "If the Security Service suspects that a potential terrorist cell is communicating with other cells, however, they will be able to track communications through IMP to locate the others even if those comms are fully encrypted."

    They can do that already, got software for it and everything, real time even. Why spend £12bn quid to reimplement an existing, and effective, system ?

    "Sure, there are wider implications for personal privacy but I don't recall seeing mass protests about the number of CCTV cameras that watch your every movement."

    So it's OK as long as no one protests is it ? And again, you miss the point entirely, an expectation of personal privacy has fuck all to do with objections to this project.

    "It's exactly the same type of data gathering as carried out by supermarkets through loyalty cards or Phorm targeted advertising that's been reported by El Reg recently.

    Knowing how people move about, communicate or even idly surf is now valuable data for mining."

    Either you are a troll, or you are magnificently clueless. That argument is so stupid that I can barely bring my myself to refute it, but just in case you actually believe it, and in no particular order : You seem to have missed the fact that people aren't happy about phorm, this is about as far from targeted advertising as sheep are from nuclear physics research, supermarket loyalty cards don't track peoples movements or intercept their communications, and you seem to be confusing commercial marketing activities with mass surveillance by the state, which suggests some rather serious mental distress on your part.

    "To bastardise the Marshall McLuhan quote, the medium has become the message."

    If, by "bastardise" you mean "take completely out of context, misunderstand, change and then use in a way that renders it totally meaningless and makes makes you look like an utter knob", then yes, otherwise, no. It's very clear from that statement that you haven't actually read McLuhan, who had nothing whatever to say about mass interception of communications. Do you even know who McLuhan was ? Prat.

  44. Anonymous Coward
    Thumb Up

    is it just me

    or isn't this simply an exercise to update the snooping kit already on the PSTN because BT is rolling out 21CN?

    Of course they'll be able to intercept mail, txt and voice (in theory) because it's all converging to IP and 21CN is all about replacing the plain old telephone system with a huge IP network, like all the much smaller OLO's have been using for years. £12b seems a realistic figure to me (for a change), you could probably thrown 21CN in plus fibre to kerb for that price especially as they've commited to all the soft stuff in the £1bn "trial" already.

    Thumbs up for a bloody good article BTW (which can be more than said for the today's coverage in the broadsheets). Keep up the good work El Reg (Chris).

  45. Anonymous Coward
    Black Helicopters

    @@MI6 grenade attack

    "Apparently, so he says, sugar dust is explosive, and you get a lot of sugar dust when filling th silos. "

    Practically any finely divided powder with a carbon content can be explosive when heavily aerated. Simple experiment: put a small quantity of dry flour in a tube (like a toilet roll tube - heh heh Blue Peter never had anything like this!) and then blow it hard across a flame. If you still have any eyebrows left you'll get the idea...

    Same thing as what happens in coal mines and why you need spark arrestors and such like (in other words, not just for the methane build up).

    So you see if you arranged for a LARGE quantity of combustible powder to be quickly distributed in an enclosed space where there was a naked flame, spark or other method of ignition... oh wait, they might be reading this post.

    Whose that knocking at my do

  46. Anonymous Coward
    Thumb Down

    snooped on the streets and now online

    Are all you BRITS now happier that everything you do is being snooped :)

    Has this made this miserable island any happier :)

    UK SUCKS it really really sucks Its turning out to be the worst cuntry to live in

    THANKS NULABOUR

    Whens the riots ?

  47. The Other Steve
    Black Helicopters

    @Luther Blissett

    For once we agree almost entirely.

    Your MoANN concept is quite scary, and also quite probable. Artificial Neural Networks (ANNs) are pretty inextricably linked with data mining, and indeed telcos and banks use them for fraud detection.

    ISTR (and it's a while since I was involved with ANNs , so this is a bit stumbling and open to correction) that the value of their use as a data mining tool is based at least partly on the fact that to become usefully predictive, you don't actually need to know how the underlying variables relate to each other, or even which ones are important, and in fact even if you are some way to this knowledge, ANNs can still predict even for types of relations that can't be expressed in term of statistics.

    If that's about right (anyone ?), then that sounds ideal for this sort of thing. It also sounds expensive, since an ANN of sufficient complexity to work on a dataset of the size we're imagining here which is expected to produce results before it's implementers are in their graves would require a substantial amount of parallel processing power.

    And of course, you'll notice that I've used the word "prediction" in there a few times, and prediction is one of the four Ps* we've been hearing about w/r/t our all new War On Terror strategy, which looks suspiciously like the one we had before.

    As you note, mining of traffic data to show connections is already done, apparently quite effectively, to link individuals and groups. What we're really shite at is prediction, and this is especially the case, if, as you also note, the admission of the fact that not every threat actor is visibly connected to Al Q means that our existing profiling tools (and profiles) are inadequate.

    Given that, you would (if you were some kind of totalitarian blow hard with a paranoid streak a mile wide) want to be able to predict which individuals that are currently not on your radar, and currently don't fit any of your profiles are likely to become threat actors in the future, and more importantly, you want to be able to generate new classes of profiles that can be applied to flag up these individuals. Hello ANN based predictive data mining.

    A sort of silicon based Pre Crime, if you will. Of course, it probably wouldn't work, and it's a Stalinist's wet dream, but those haven't been deterring factors in any of NuLab's other IT projects.

    *Those would be Prevention, Prediction, Protection and Pursuing, ISTR.

  48. Charlie Stross
    Joke

    That weird sound ...

    That strange noise in the distance is <a href="http://en.wikipedia.org/wiki/Erich_Honecker">Erich Honecker</a> wanking furiously in his grave.

  49. Anonymous Coward
    Happy

    @This is sorting, not collection

    I agree with every word. Any attempt to be obstructive will just cost more money.

    I don't think it's a difficult thing to do this matching though, as I designed a similar system a few years back. You don't know what you've got, but you know what there is, so you have a tower of reference data, and you do recursive offline matching to the known, with hundreds of appended foreign keys (some of them hierarchical in nature,) in your dirty data.

    This isn't rocket science. Any stupid moron could design it.

  50. Strappy
    Happy

    @ The Other Steve

    Oh, well done. You've successfully taken my comment, broken it down and responded to each point in turn while completely missing the fact that it was a response to the paranoid comments above.

    quote> We're told. But you've missed yet another pint, although you're by no means alone in that, the name of the project is the "interception _MODERNISATION_ program", which rather suggests an upgrade of present capabilities. If you think that present interception capabilities don't extend to content monitoring than you're living in la la land.

    "it's going to map the connections between people. Only the most naive terrorist would send an email containing such obvious trigger words as "bomb" and "assassination" and those emails would doubtless be flagged and filed..."

    You have just contradicted yourself, postulating here that content will indeed be scanned. <end quote

    Thanks for showing how quoting out of context can change a message. I pointed out that there are plenty of interception systems already so it's hardly a contradiction.

    quote> There have already been high profile cases where people were detained and charged based on such things, or don't you watch the news at all ? <end quote

    I'm aware of cases such as you mention but I don't recall any of them coming about from an email about the weekend's football results that has "BOMB" at the end to "game the database" (for the sake of an example).

    quote> They can do that already, got software for it and everything, real time even. Why spend £12bn quid to reimplement an existing, and effective, system ? <end quote

    Because it's easier to have it all in one place rather than demand access to logs from multiple ISPs, perhaps?

    quote> So it's OK as long as no one protests is it ? And again, you miss the point entirely, an expectation of personal privacy has fuck all to do with objections to this project. <end quote

    Out of context again but what the hell, you're on a roll.

    quote> "It's exactly the same type of data gathering as carried out by supermarkets through loyalty cards or Phorm targeted advertising that's been reported by El Reg recently.

    Knowing how people move about, communicate or even idly surf is now valuable data for mining."

    Either you are a troll, or you are magnificently clueless. That argument is so stupid that I can barely bring my myself to refute it, but just in case you actually believe it, and in no particular order : You seem to have missed the fact that people aren't happy about phorm, this is about as far from targeted advertising as sheep are from nuclear physics research, supermarket loyalty cards don't track peoples movements or intercept their communications, and you seem to be confusing commercial marketing activities with mass surveillance by the state, which suggests some rather serious mental distress on your part. <end quote

    I used Phorm and loyalty cards as an example of tracking data, not as a comparison to a major government project - you drew that conclusion all by yourself, which suggests you have an agenda to pursue here but like most people who respond through vitriol you are unwilling to discuss the issues with anyone who might disagree with you

    quote> "To bastardise the Marshall McLuhan quote, the medium has become the message."

    If, by "bastardise" you mean "take completely out of context, misunderstand, change and then use in a way that renders it totally meaningless and makes makes you look like an utter knob", then yes, otherwise, no. It's very clear from that statement that you haven't actually read McLuhan, who had nothing whatever to say about mass interception of communications. Do you even know who McLuhan was ? Prat. <end quote

    If I'm able to quote him, I must be aware who McLuhan was, mustn't I? Or did you think I just got lucky on a random quote search?

    Of course McLuhan wasn't talking about mass interception of communications, that's why I flagged the quote as bastardised. The point of my comment was that the actual movements of people has become valuable data, not just for governments but for search engines, social networking, studies of public response, the list goes on. Any use of the internet leaves a trail of IP addresses that is useful for all sorts of purposes as touched on in both our comments.

    I know you won't agree with anything I've written here, chances are you're probably away hugging yourself at just how clever you are. Well done - you got angry on the internet. Have a sweetie.

This topic is closed for new posts.

Other stories you might like