Running queries on the HMRC database fiasco When it comes to talking about last week's data loss by the HMRC, I was told not to use precious words outlining my feelings of rage and bafflement that a government body can be so cavalier with so much data because, presumably, we all feel the same. So I will simply note, for the record, that my gob has been totally smacked …
Thinking about ID cards: I was talking to a friend who is entitled to an Irish passport.
He/she said that if ID cards were introduced, the whole family would immediately activate their Irish citizenship.. and refuse to carry ID cards.
What proportion of the population in the UK would be able to do the same?
>> The NAO (not having a mainframe) then passed the database to KPMG to process. -- Anonymous John
That is an interesting addition to the story. The only reasons that I can see the NAO would need mainframe to handle 2 CD worth of data are (if they only wanted to process 100 random records):
1. Their desktop PCs are steam powered.
2. The data is in some sort of raw format which can only read by an obscure DBMS, which only runs on a mainframe.
In todays ultra-technologically advanced world where fighter pilots can shoot down their targets using in-visor overlays and a nod of the head, where infantry can track down the scrambled and secure cell phones of foreign dictators, where apparently Israel can sneak past Syria and maybe do *stuff*, where our personal data is at its most vulnerable as the Government seeks to consolidate that data and store it in one place..
In this society, the latest scandal isn't that our data was hacked and sucked off into a torrent file, there was no IT security failure in terms of hardware breaches or software cracked, there was no inside man handing out passkeys or ID badges or flicking power switches and blacking out surveillance. None of this. The Government simply put the data onto a CD and put the CD into an envelope.
I suppose we should be grateful that the NAO didn't just ask Alistair Darling to empty all our bank accounts and send them the money as from the sounds of it he would have done.
"Hey, exceedingly junior scapegoat, stuff that money in that envelope and send it in the post"
"But sir.."
"Now, now young chap, time is money, on with the stuffing!"
I don't believe that will help them at all (unless they move back to Ireland) as the Government plans to force *everyone* living and working in the UK to have an ID card. About the only people that will be immune to this gross invasion of privacy will be foreign diplomats and the Queen.
The whole ID card thing is truly frightening because of two opposing issues:-
1. Just like any other Government IT project, the whole thing will be an absolute disaster and the highly sensitive and personal data the database contains will be about as secure as a chocolate fireguard.
2. The Government believes ( or is, at least, spinning that )the ID card system will be secure and infallible.
Put those together and the opportunities for miscarriages of justice are immense.
"The NAO admitted that later. No reason at all for HMRC to know what the NAO did with the disks."
NAO commentator also went on to say that the data was HMRC's, and that HMRC had a duty to care for the data. NAO therefore consider passing on the data to a third-party for processing to be 'business as usual'. It's possibly true that HMRC had no knowledge that NAO was going to pass it all on, but NAO have also been irresponsible here since it was told by HMRC management that the data would not be desensitised.
Oh, and on that matter, 'desensitised' is not the same as removing fields, and may mean recoding certain parts of citizens' records in a way that allows whatever statistical analysis NAO had planned to actually be carried out. It takes a short time to remove entire fields from an export file, but a long time to replace information that preserves uniqueness of, say, the NI number or the postcode/house number combinations without giving away information that could be used to directly link an individual - as I understand it, NAO was auditing the HMRC, not trying to find benefit cheats, so they needed to know that HMRC was on top of the potential for fraud, rather than being able to hand back info about what X, Y or Z were up to... (apologies if they are monikers anyone here uses - I was going to put 'Joe Blogs'...!)
I think here there's another issue - govt depts not being clear with each other.
And there's one more cost to the country that was missed from AC's list - I've already recieved a letter from the acting head of HMRC to tell me to not panic. - 7.5 million letters at 20p or so (probably less due to the bulk mail - IIRC it could be 16p each after discount) has already been spent on spin.
> What if some government-paid twit had decided to store the data in XML format, as is the fashion?
Not likely. The data should be nicely record-oriented, so using XML is Bad Practice, not to mention unwieldy. It is most likely in some RDBMS.
> That would mean it was hard to strip out the bank details, etc.
That's not a "hard" that I recognize. You can use XML extraction languages like XQuery to get at the data, or you can roll your own with some Perl, a task in the order of an afternoon's work (until everyone is satisfied).
Well, at least KPMG's deleted files are safe as houses in the "Recycle Bin" on their system. I'm sure they emptied it after the deletion to make sure it was unrecoverable.
I sure hope they ran an multi-overwrite and put plenty of garbage over the freed-up disk space after the deletion?
Surely such an outfit would know what to do to make data recovery very, very difficult when the next refresh sends their current kit off for auction!
What's the betting there's a huge team working on the 'stategic' solution of rthe database, with the new system due in "6-12 months". Meantime all you have a creaking access db to do the days work, the DBAs telling you you can't have anything, ever and the strategic system producing nothing but powerpoint.
Losing the ID Card database would be even worse than you describe. No need to reverse engineer the fingerprint codes - just look for any that are close-enough to your own. Large scale trials of the US-VISIT IDENT system found about a 0.1% false match rate, so in a database of 40 million adults a typical crook should find around 40,000 people to rip off at leisure (complete with names, addresses, passport numbers and all).
I'm surprised no one has spotted the obvious inter-departmental politics in the response that removing unnecessary data was too costly. The NAO website says "The National Audit Office scrutinises public spending on behalf of Parliament.....Our work saves the taxpayer millions of pounds every year."
The response that removing unnecessary data was too costly was obviously a political response. "If you're so worried about costs to the public, you deal with it." It was obviously meant as a polite 'up yours' to the idea of creating more work and hence costs (however small) in order to perform the audit. In other words, a politically expedient way of avoiding doing extra work.
Yes, I am _really_ annoyed that the NAO has lost yet another set of sensitive data. A similar thing happen with employee data earlier in the year.
But the big question, which no one seems to have mentioned, is why are the allowed to request the data in the first place. Isn't the Data Protection Act to protect and prevent the disclosure of personal data to unauthorised bodies.
In local government, we're not allowed to share data with other departments or authorities without notifying the named persons in the data. You need there consent. Why is it different for central government??
As someone who knows a little more than nothing about encryption I'd like to point out that even if they'd encrypted these disks you should still have been worried.
This data is going to have significant value for many years. In fact it will only start to make good money for the bad guys in about 3 years, and then onwards for a lifetime, what's that, maybe another 80 years more. If bad guys have got them they'll probably sit on them for a good few years before even starting to use them.
Cryptography is always advancing, and so is the speed of machines. Encryption systems in use today will be broken eventually, they always are. These disks have a significant value, and will continue to do so for a long time. It would be worth the time and money for the bad guys to break the scheme in use. (I now look forward to myriads of posts about how hard it is to crack the current encryption schemes. Yes, currently it is hard, but next year it will be easier, and in ten years: probably trivial.) Considering the governments wherewithal on security, I doubt they would have encrypted it properly anyway, even if they'd tried.
The issue is not the use of CDs, the posting in the mail, or the lack of encryption, the issues are these lunatics thought it OK to send a large quantity of that quality of data about, as it exhibits a monstrous level of cluelessness, and that people so cavalier are even allowed through the gates, never mind given positions of authority.
Well, having worked for KPMG, I can say without doubt that there are people there who WOULD know all about data security, and how to securely delete a copy of a file. Their Forensic Accounting department, for example, frequently had to recover "deleted" and even "overwritten" data. They were quite the impressive bunch. Probably downsized in the interim, of course.
On the other hand, my abiding memory of KPMG was being called to a senior partner's office because he was having trouble opening a Word document. A short explanation of the difference between double-clicking (cli-click) and clicking twice (click click) was required.*
I rather fear that this would be the level of numptie with which these data would have been entrusted...
* Of course I can feel as superior as I like, but at the end of the day he was "earning" six figures for not knowing how to double-click while I, with all my wit and sophistication, struggle to support a family and a mortgage. Who has the last laugh?
If a DataBase had Total Information Awareness of a Citizen's Needs for the Future he has Seen to Share, IT would Allow for Government Payment of Public Money to a Citizen who has Shared Everything for Transparency to Liquidate Valid Future Costs/Past Expenses.
And all apparently for a measly seven seven figure sum. QuITe obviously the powers that be are not au-fait with the Power in Miners of Rock.
And boy, are they in for a Pleasant Surprise Package? Not 'arf.
"Reality leaves a lot to the imagination." .... John Lennon.
The problem was summarised very well in the first comment. EDS is to blame. Why is nobody pointing the finger at them?
The bigger picture is that these problems will continue to occur because the incompetent/corrupt/stupid/lazy buffoons that award the contracts for these systems only seem to rely on the response to one question: "Has your organisation done anything like this before?".
Only the usual suspects can answer "yes" to this questions, so only they get chosen. It doesn't seem to have occurred to anyone to ask "How badly did you f**k it up last time you did something like this before?"
Thanks David S for the insight. I kinda guessed that would be the situation. BTW – knowledge is of far greater value to the world than money, but harder to pay bills with :-(
However, how about the ability to designate certain fields as “confidential” in such a way as to lock them down, make them non-printing or non-exportable or whatever. I mean like ”Admin” rights allow, or deny shares, edits and read/write actions on files. I imagine that the makers of decent software have thought of this one? Kind of a built in automatic filter that simply won't let the entire data set be copied/cloned without the intervention of an authorized “owner”, of suitable seniority and nous, who would have to carry the can if things went wrong because their details would be embedded in the data set. Of course it all comes down to human actions and ability levels and there's usually always a “work-around” somewhere I guess.
We are all passengers now on the information super-highway but we don't expect the trolley-dolly to be flying the plane when we are at 32,000 feet over the Atlantic! We would all refuse to fly, I suspect, if we thought for a moment that might be the case. But how can we refuse to be swept along at 100Mbs, hurtling towards inevitable disasters, such as that of HMRC, because there's no-one at the controls actually! Scary, stop I want to get off.... now!
You're all talking about ID cards as a Bad Thing (tm), which in its current incarnation and planned use it surely is.
However, the issue and control of those lovely National Insurance/Health/whatever numbers has been so totally botched that anyone who has looked at it over time has declared it non-fixable.
There is a HUGE amount of benefit fraud performed by the use of the cracks in the issue system, and thus one of the non-Orwellian drivers behind the IDcard was to redo the body numbering from scratch. It doesn't excuse the rest of it though..
- I wonder how many times data has been sent on CD's and arrived safely.
- I wonder how many times data has been sent on CD's and forgotten, never to arrive and just getting lost in the internal mail.
It is good that this information has gone public. It could of been very effectivly covered up. For that we should give our Government some credit.
Also, out of the 1000's of data files that get exchanged within government, it was going to happen one day. This is the price we pay for storing sooooo much data in one place.
"I imagine that the makers of decent software have thought of this one? Kind of a built in automatic filter that simply won't let the entire data set be copied/cloned without the intervention of an authorized “owner”, of suitable seniority and nous, who would have to carry the can if things went wrong because their details would be embedded in the data set. Of course it all comes down to human actions and ability levels and there's usually always a “work-around” somewhere I guess."
Scott,
The owner of any decent software would intentionally embed all relative details in the data set so that it runs to specification. In fact, it is quite naturally included in every thought that we share/line of code that is written and decent software has broad enough shoulders and a thick enough skin to carry the torch rather than harbour any thoughts on carrying a can.
It is a subtle failing in programming, which may be intentionally placed there, to have doubt hinder ability thus maintaining a Moribund Status Quo Logic. A gift from you know whom.
SeXXXX IT, Billy Boy, Breathe some Life into the GAIme. In the SurReality of Virtual Space though, is Power Directly Proportional to Proxy Ethereal Control of MindSets with an All Pervasive and Addictively Persuasive Seduction ..... in Order to Guarantee Positively Reinforcing Results.
Posting Anon for obvious reasons.....
Worked for the Met and still have contacts inside at High Level in Empress State/Cobalt Square/NSY.......
I was looking at Virtualisation for a project at my current employer and mentioned to mate about how good it would be for their department to Virtualise the servers...
Apparently they are not allowed to even think about changing/moving the databases and/or any of the servers they reside on as the contracts are signed for over 10 years..... Any changes they make have to be approved by CapGemini and also carried out by their engineers. The costs involved are staggering......
They are having a refresh of the systems there and the desktop systems they use for email ("AWARE") are being upgraded. Long story short they also have desktops sourced internally and an engineer from CapGem, rudeboy type with matching ringtone, was meant to be installing replacement machines where nessecary and dismantled a machine because he didn't understand why the machine wasn't working on AWARE and left the machine POST erroring due to removal of the memory and went home. Suffice to say mate found machine and it happened to be a machine that engineer wasn't authorised to touch. Can they do anything about it.... Can they eck.... Oh and if you happen to pay UK tax you paid for that idiot to spend the day breaking stuff.
P.S In case any are interested....The issue with the machine is that it was plugged in underneath the desk into a KVM so the user could switch and only use one monitor. The one he was meant to refresh was under said monitor. Oh and the engineer/moron gets paid over 30k pa for this.
May seem strange; but I don't believe for one second these 'disks' have been 'lost'; I don't even beleive they were sent in the first place. This ordeal is far far 'too' stupid for the Government.
I beleive this is a way to promote ID Cards, so people buy into it in a bid to 'protect our personal details'!!
They know exactly what they are doing, and we all just walk right into it; everytime!
[/conspiracytheory]
"Apparently they are not allowed to even think about changing/moving the databases and/or any of the servers they reside on as the contracts are signed for over 10 years..... Any changes they make have to be approved by CapGemini and also carried out by their engineers. The costs involved are staggering......"
An Inequitable and Unfair Slave Contract, AC,......and probably Illegal/Criminal for it would appear to guarantee Non Competition Complacency/Gravy Train Riding rather than keeping evryone at the top of their game.
Paying for Failure ... the New Labour Way.
Apparently 7.5 million letters have been sent out (giving Royal Mail £1m they badly need?) and 25 million records were lost. So maybe 18 million children and 7.5 million parents (mostly mothers) are involved, and the mothers are the ones whose bank accounts are at risk.
Has any journalist or politician spotted this relationship?
Fortunately I have no young children, I am retired from IT and I live in France, but I sympathise.
re Baby Boomers
Mr A Coward says:
"At least in the US, when the Baby Boomer generation is all dead, we will finally have politicians who have SOME grasp of technology."
And we will also have in power a generation where there is widespread belief that Earth is 6,000 years old, that the US never sent spaceships to the moon and that science is a godless conspiracy to lure ordinary folks into Satan's lure.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
