Kept sectret from the public, or from the target government ?
It's not clear from that article whether the calls were tapped with or without the target country's support.
The NSA is recording all of the voice calls in one unnamed country and keeping those recordings for 30 days at a time as part of a previously undisclosed rolling wiretap programme, according to leaks recently published in WSJ. Millions of voice "cuts" are extracted for long time storage as part of a system called MYSTIC that's …
"....if its the UK....." Don't be silly, the UK is probably waaaay down the list. My bet would be on Pakistan as top choice, followed by Qatar, then Saudi, UAE, North Sudan, North Korea, maybe Cyprus (fave hangout for Russian mafia, arms dealers and exiled Palestinian and Lebanese "freedom fighters"), or even Switzerland (not a NATO partner and full of dodgy bank accounts) before we even get round to any large European countries.
Every call recording system I've ever worked with have used WAVs
Only for ease of use, plenty will do smaller MP3 and even better, many with do vox files, which are tiny.
So long as they are not capturing DTMF tones, as they wouldn't want to fail any compliance laws would they?
Well, supposing the country is the UK, a quick Google search fetches up the number of 132 million phone calls made every day here. Suppose they all last for an average of 10 minutes (not everyone can match my mother's phone habits, after all), and it's stored as 8kHz 16-bit PCM (8kHz is what the POTS is designed to carry, being sufficient for human voice) then over 30 days you're collecting 1.32x10^8 x 10 x 60 x 8x10^3 x (16/2) x 30 = 1.52x10^14 bytes required to store it all for 30 days. 152TB. It's not peanuts, exactly, but surely the NSA can manage better than this?
And before someone leaps in, yes, MP3 or Vorbis or whatever could reduce that a bit, but bear in mind that they work by throwing away frequencies that aren't interesting, and you've already thrown away 80% of the audible frequency range by encoding it as 8kHz PCM; you're not going to get the same compression ratios that you managed with your CD collection.
@tom7 RE: 8khz
I had actually thought it would of been a better quality and or possibly compressed into another format. that way if needed to they could analyze background data etc etc
But i suppose that 150TB isn't much at all when your thinking about the volume.
But then add on top of that replication etc etc maybe it does add up quite quickly.
I make it out, on the given assumptions as:
132*10^6 calls/day,
600 seconds/call,
8000 samples/second,
2 bytes/sample, and
30 days/month
so 1.32*10^8 * 6*10^2 * 8*10^3 * 2 * 3*10 = 3.8*10^16 bytes, which my calculators assure me is 34,560 TB, rather larger than the 152 TB given, but less than the 152 petabytes that the calculation shown actually gives. Still doable, though, with about 8600x4TB disks.
The unnamed country probably is not the UK. I can think of a number of likely ones, none of them English speaking, European, or American (either North or South).
That's 360TB. With half-decent compression -> 100TB.
So uncle sam needed to light up multiple (like FOUR) 100Mb cables between his victim and back home without ANYONE there being a little bit suspicious? That must have cost a fortune.
Seems likely the country has a complicit government to have helped out with the logistics and the financing. Pakistan?
"number of 132 million phone calls made every day here. Suppose they all last for an average of 10 minutes (not everyone can match my mother's phone habits, after all), and it's stored as 8kHz 16-bit PCM (8kHz is what the POTS is designed to carry, being sufficient for human voice) then over 30 days you're collecting 1.32x10^8 x 10 x 60 x 8x10^3 x (16/2) x 30 = 1.52x10^14 bytes required to store it all for 30 days. 152TB. It's not peanuts, exactly, but surely the NSA can manage better than this?"
Phone CODECs are a bit different to the normal (linear) kind.
They use a non linear sampling law (Merkins use a "mu" law, the rest of the world a different one). Basically that compresses 12 bits of dynamic range to 8.
The question is do you have to expand this to linear forms to do analysis or can you keep it in the 8 bit format?
For a worst case calculation, assume everyone in the UK is on the phone say 12 hours a day 365 days.
It's a big number.
But not big enough that the cost will deter the true data fetishist.
Following on to that, obviously you're not going to be doing CSI esque analysis of the audio after it's been degraded that much, but perhaps they view the speech part of the recordings as worth having even without detailed background noise. Important targets could be stored at a higher quality.
I realise you're only joking but Stella in the UK is actually produced in the UK and tastes like piss compared to the stuff that is brewed in Leuven, Belgium. I used to have my French ex bring me back some of the real deal, amongst other weird and wonderful brews, every time she went home to visit her parents.
That all depends on your definition of evil as well...
If it's a test they would surely want to test it on a high stress target, so they could spot any flaws.
Further though, if the Washington Post knows the country, what kind of journalism is it when you don't reveal it simply because your nation says so? Perhaps there is no actual story and it's simply a plant in order to make everyone else think the NSA have greater capacities than they really do?
10/1 it is a nuclear armed Islamic country with a reputation for hosting terrorist training camps in ungovernable badlands, a widespread fundamentalist insurgency , strong cultural and economic links to one of the big five, and a local intelligence community who are as likely working for the opposition as the home team.
"10/1 it is a nuclear armed Islamic country with a reputation for hosting terrorist training camps in ungovernable badlands, a widespread fundamentalist insurgency , strong cultural and economic links to one of the big five, and a local intelligence community who are as likely working for the opposition as the home team."
London, then?
Maybe they just did somewhere small as a proof of concept? The Vatican for example!
Other likely spots...
Rockall - highly contested land mass that's only ever been populated by extremists
Colombia - war on drugs and all that..
Israel - lets keep an eye on those crazies, they could kick off with any of their neighbours at any time...
It can't be that hard to work out which countries threaten our 'security'. We're looking for a non-Christian, Socialist country that recognizes women's reproductive rights, supports higher education and health care among all segments of the population.
Holy fucking shit! It's Canada! We've spent billions of dollars to capture the voice traffic of our neighbor, biggest trading partner and #1 supplier of sandpaper! This is a sad day indeed. I bet they won't sell us sandpaper anymore.
Speaking as a Canadian, I feel that if you suck up our phone calls ("hello mum? Yeas, freezing cold here too..."), you must prepare for the Wrath of the Canucks, and face a future of unsmooth wood surfaces. Sandpaper embargo!!!!!!
We will hear your screams. We will not care.
...you must prepare for the Wrath of the Canucks, and face a future of unsmooth wood surfaces. Sandpaper embargo!!!!!!
Do this and be prepared for a dirty, hit below the belt, ruthless war! We'll start by deporting Justin Bieber back to Canada! Before too long, those friendly little Canucks will be doing everything they can to make themselves deaf and blind! You'll never recover from that fallout MUHAHA!
To frustrate Americans and ship stuff to the States in wooden containers, using Robertson Headed screws is really effective in winding them up.
Robertson screwdrivers are extremely hard to find down there.
Note: A Robertson, also known as a square screw drive has a square-shaped socket in the screw head and a square protrusion on the tool. Both the tool and the socket have a taper, which makes inserting the tool easier, and also tends to help keep the screw on the tool tip without the user needing to hold it there.
When Henry Ford tried the Robertson screws he found they saved considerable time in Model T production, but when Robertson refused to license the screws to Ford, Ford realized that the supply of screws would not be guaranteed and chose to limit their use in production to Ford's Canadian division. Robertson's refusal to license his screws prevented their widespread adoption in the United States!
"Note: A Robertson, also known as a square screw drive has a square-shaped socket in the screw head and a square protrusion on the tool. Both the tool and the socket have a taper, which makes inserting the tool easier, and also tends to help keep the screw on the tool tip without the user needing to hold it there."
I'm not sure if it's genuine Robertson, but a few of those screwdriver tip sets I have aboot the house have square-drive sizes, along with a few tips I've not seen before. Never seen a Robertson head on any item I own though, that seems to be taken over by Torx.
Here in the US, Robertson drive fasteners are also referred to as 'clutch head screws' and 'trailer screws', because their biggest use is in mobile homes where the 'drywall' is held in with plastic cames screwed to the studs. The heads of the screws are exposed to the room and the whole effect creates a very riveting (Ha!) appearance.
If you're into that kind of thing, there is a fantastic book called 'One Good Turn' (forget the author). It's a small book, but full of big examples of the history of threaded fasteners. The book is available, cheap, on Amazon.
Ford is indeed responsible for the 'intersecting drive' (Phillips and similar). As I suppose you may already know, but Ford wanted to make the fasteners internally, not buy them from the manufacturer (Robinson). Robertson was concerned that if he licensed the manufacture of the screws that Ford would not only steal the design via legal shenanigans, but be able to produce them at a much lower cost as well. Those are very valid concerns, even with today's technologies.
An upshot in the way all that went down is that Ford's tooling staff would go on create and refine a lot of fastener attributes still used today as well as perfect the high volume, high tolerance manufacturing techniques that, quite literally, hold the modern world together.
Bad choice, Canada. We talk our heads off and have conversations almost morning to night.
Back in the day, I was a communications contractor for Maritime Tel and Tel in St. John's, New Brunswick, and Mondays were the best. We would put the testers headset up on the switching centre public address then we would touch the line-side jacks to the field of sockets until we happened upon a juicy conversation.
The best were when two females were discussing their weekend seductions - real laugh.
In the large centres such as Halifax or Moncton we would tap into the operator headsets in the off chance they were comparing notes with fellow operators.
Unfortunately, large automatic switches were introduced which started eliminating many operators. What a pity.
"Bad choice, Canada. We talk our heads off and have conversations almost morning to night."
Not if your staying in the Holiday Inn, Ottawa and using some flavour of VoIP, you're not - you can't get much past "Can you hear me!" before the WiFi drops out again, and again, and again. Or perhaps the NSA were uploading in the room next door.
Maybe burning down the White House isn't quite as forgiven as you'd thought.
Oops, that's Obama caught telling porkies.
and "last year’s secret intelligence budget named five more countries for which the MYSTIC program provides “comprehensive metadata access and content,” with a sixth expected to be in place by ... October (2013)"
Here we come - all the worlds telephone content recorded for ever!
Geeks were starting to worry about our internet era being lost to the archeology of future civilisations - obviously the NSA Utah is just rather enthusiastically following on the work of Sir Flinders Petrie, it's not like knowing everything about everyone could ever be used for harm - it's just 'truth.'
So, do you reckon Obama knew that they'd been up to stuff like this, and chose to lie (hoping that it wouldn't be revealed), or that he was told by the spies "nah prez, course we wouldn't do that", and thought he was telling the truth?
I cant imagine in this day and age (especially after the first lot of NSA data breaches was revelaed) that he would have made such a definitiev statement if he knew it was bollocks. He would have used weasel words, and specially formulated statements that said something without ever denying the possibility of anything.
This sounds to me like he said what he thought was the truth, and which now turns out not to be. Mind you I could be wrong, and when he said those sorts of things what he meant was that they dont do it to USAians. Since Yanks dont usually consider the other inhabitants of the world to be people I wouldnt be surprised by that turn of events...
People,
I live in the USA and we ARE recording everything you say on all your phones. The British Intelligence people are recording everything WE say.
Unfortunately, that makes all this shit LEGAL. If our own people were to record our own phone calls, it would be illegal.
OOooopps, too late. Black helicopters are coming.........
... then you will not mind if we store all information about your life and everyone else in your country. Just in case we need it in the future to see what happened in the past. We won't look at it (looking at it in NSA jargon means that they have collected it, a human not looking at it means that it was not collected). Spy language is tricky stuff to get your head around.
So storing data about everyone in a country is legal in the US, and because the US are the world police it is legal everywhere else as well. America (and GCHQ in the UK) own the planet. Trust them they know what they are doing, and they have guns :)
Well, not anymore.....
Interesting though that they chose Utah, not too far from one of the other biggest stashes of information on the human race which is bunkered in a mountain owned by the Mormon church. Maybe the Mormons have done some successful door knocking in Fort Meade.
So the NSA doesn't do intrusive spying, they are just looking for the terrorists. And the people who are in contact with terrorists. And the people who live in the same house, city, country, or world as those terrorists. But it's OK, because they only collect metadata. And all your emails, telephone calls, and video. In public, private and, in particular, if you try to turn off your computer/smartphone microphone or camera.
Now why they can't catch any terrorist when they already subverted all of humanity is a bit of a mystery? Clearly time for some 'enhanced' data collection to fix that. Everybody please make an appointment with your local secret service office and don't forget your bathing suit and towel. And remember, you have nothing to fear if you have nothing to hide. Nobody innocent was ever harmed by a few questions and only guilty witches ever drown when submerged.
"Handling and transmitting bulky voice files acted as a major snag in putting together MYSTIC, at least in its early days. Around a year after MYSTIC went live, a programme officer wrote that the project "has long since reached the point where it was collecting and sending home far more than the bandwidth could handle," the Washington Post reports.
Similar capacity ceilings have cropped up across a range of NSA collection programs, a factor that explains the spy agency's move to cloud-based collection systems and the construction of a massive “mission data repository” at a new facility in Utah, the Washington Post adds."
So, assuming "mission data repository" is final destination, all data still needs to end up in one place. In that case any cloudiness is not likely to help at all since ultimately all that data (and in case of voice calls unlikely to compress that much) still needs the network bandwidth to the repository.
They record the entire world's facebook and gmail, watch lovers doing rude things on yahoo chat and even monitor your angry birds activity. And they have spooks running round WoW looking for 'terrists' too.
But an airliner full of 250 people can vanish and 12 days later still be missing.
It's good to see they have their priorities straight after 911.