Removing innocent people's records from the DNA database will cost almost £5m, the House of Commons was told yesterday. MP Diana Johnson asked crime prevention minister James Brokenshire how the batch loading system for getting records onto the national database worked. He explained that DNA samples are first converted into a …
That seems very cheap compared with the cost of the database in the first place
We should get Blunkett and pals to pay for it, personally! For having the nerve to think keeping profiles for ever would be acceptable.
Maybe they should have a word with someone who knows some SQL, I'm sure they could do a few joins and delete the offending records.
How much could they have saved
By not collecting outrageous amounts of DNA profiles on totally spurious grounds in the bloody first place?
Price of justice, I suppose
Get it wrong and suffer the consequences. Isn't that the precept of the law?
is that for all, or each?
as the title says. if it's £5m to remove every innocent person's DNA then fair enough. if it's £5m per person to remove their DNA then I might suggest that someone needs to go back and look again at the bill.
Surely it's a case of:
DELETE FROM [DNA] WHERE [crime_committed] IS NULL
I'll do that for £2.5Million...
Because that would imply that the database was well designed.
Given that this was a government project, it seems unlikely.
You believe a government project was set up sensibly in the first place? What about support and maintainence costs, can't make them stupidly high if they could pass it on to any old schmuck!
...you forgot to add VAT. :-)
But your point is valid....for such a simple procedure, how the fuck does this cost £5m????
Does that also mean that *adding* data to the table(s) cost £5m too???
better than that...
DELETE FROM [DNA];
SQL? What SQL
What makes you think this database is relational?
The only use of the database is to compare DNA profile X against all other profiles and print out the list of matches. This doesn't need a relational database, especially as the natural key, the DNA 'donor' name, isn't needed to do the search: it is only needed to delete profiles and that was never a requirement.
Given all the above, why pay for a DBMS when you can simply stuff files containing the profiles into a RAID 5 filing system so they won't be lost if a disk dies and because reading from it is bloody fast. Job done, and for minimal cost and zero thought except that putting 1000 profiles in each file will save file open and close time during a scan.
#rm -rf /*
do it? less than £5 million/character.
I'd do it for half the price.
('course, Id probably need to be of non-UK origin to pass the stringent security and P.V tests, but...)
Sorry, I've never used that command. I'm told it's surprisingly effective. Especially (given tomorrow's date) to leave it - as a text - on the BOFH computer first thing tomorrow morning...
DROP DATABASE [dbDNA];
how the fuck does this cost £5m????
£4.28 to write the code
£118.72 to run the code
£4,999,877.00 in consultancy fees
Someone didn't think ahead.
What if we need to delete a record?
Ahh we'll never want to do that.
Yeah but what if.
Look just make it a pain in the arse so if we do need to do it it'll cost a fortune.
~~~Several years later
You didn't make it expensive enough >.>
Verity Stob explains
"The best way to avoid risk is by advising that any activity is technically impossible for reasons that are far too complicated to explain. If that approach is not sufficient to halt the project, then the engineer will fall back to a second line of defense: "It's technically possible but it will cost too much."
WTF are they doing ?
Each profile is added to a "batch" and loaded to the database with 1,000 others?
I'm sorry, but this really is technobabble at it's worst.
Does no one else think this is a crock of the highest order ? Mind you, given the perfect storm of technical illitereacy and journalistic credulity, I shouldn't be surprised. Who explained it to Mr Brokenshire (I urge readers to check his official mugshot out) Stephen Fry ?
Must be a massive amount of DNA collecting going on
To input DNA records or samples in batch sessions. Why batch only 5 or 10 at a time? Probably batches contain a couple hundred "contacts", probably done every 2 weeks. Not sure if that is representative of the level of relevant crimes, but I'm just pulling numbers for effect.
The collation of some number of contacts' files and their entry makes me wonder if file errors or contamination can let some suspects off and convert innocent suspects into incarcerated individuals.
And, what the hell is this batch entry got to do with isolating individual records. If a set of exonerating DNA is placed in the database to "extract" a deep-cover operative who needs a convincing case file or rap sheet, then it means that finding this anonymized record will be a major BI*CH to do so and not expose other records. Records requiring privacy sensitivity should be discrete and not commingled in such a way that deleting or viewing a specific record destroys or exposes or manipulates another record.
Either they have a horribly designed database, or the explanation is a crock of shit, or the interpretation of the explanation was maligned somehow.
Didn't the database design team know of CODD's rules, atomicity, and a slew of other best practices or database "laws" that help frame the design of ANY database. Even if it is object oriented, there still must be a valid concept of data atomicity, row and column locking, audit control, recovery, and so on.
Speaking of recovery, what about all of their backups. If there are 10s or dozens of off-site archives not yet purged, how do they know they will retrieve every last cartridge, disk, or other media and the logs that make references to that to-be-purged DNA? Do the references, too, need to be purged? If their logs are incomplete or inaccurate, they will probably never successfully remove each and every should-have-never-been-filed DNA record. If any of it traveled over a wire, across Wi-Fi, or into a cloud, even if encrypted, anyone knowing of the backup schedule could sniff and hope to crack the content encryption at a later date.
Shows that compliance was never intended.
In other words, the system requirement spec explictly left out any process whereby a sample and it's records could be deleted.
That's why complying with the law is so expensive.
What was the plan when someone, everone, dies ? Was the database to expand forever ?
And what was the intention when it was found that an error, or deception, occurred at collection so that a non person was recorded ?
I suspect ...
... that when everyone dies, recovering disk space will not be a priority. In all (or slightly more) seriousness, dead people should stay on the database anyway, since it's possible they committed unsolved crimes and their records may exonerate other suspects in the future.
Also, at that point, the crime of storing their DNA for (probably) no good reason has become victimless.
IIRC the point of retention was FAMILIAL...
...matching. Sure there was some hope that a person of interest who proved innocent "this time" might be kind enough to leave a sample behind if he ever does offend.
However, that is small potatoes compared to the power of mining a comprehensive population wide sample of DNA. Even a relatively small sample would be enough to start getting direct hits on a parent, child, sibling or cousin. Once the number of samples has grown to a suitable representative size, it would soon become possible to say with some confidence: "We're looking for someone who's mother is a Kerry County O'Rourke and father is a Dun Edin McCleod." Which in turn requires a simple "hatches, catches and dispatches" lookup to resolve. Case closed.
Except with such a comprehensive database it would also be possible to frame up just about any member of the encompassing population simply by salting the scene with their DNA.
Big Brotherish enough thankyou that, even limiting the database to convicted criminals, still allows the "over the odds" apprehension of crime "families" with less than thorough police work.
Sympathetic (Voodoo) magic for real, where fingernail parings, hanks of hair, your blood and even your poop can be used to destroy you.
There is hopefully one silver lining. Technology is advancing at such a rate that spoofing advanced forensics like this is within reach of enough people, that conviction on forensics alone will hopefully soon assume a similar status to "only CIRCUMSTANTIAL evidence" as it properly should.
"We have compared the sample found at the scene of the crime and we are looking for a relation of miss Eve Mitochondrial who may have lived in east Africa some 200,000 years ago"
Are they saying hash the DNA profile, add it to a file with a thousand other hashes and then store this in a blob field?
How the bloody hell can they *find* anything, let alone delete it.....
With a very very expensive, complicated and messy front-end connecting to this massively proprietary database that is keeping some third party (for want of argument, lets guess Accenture or IBM) in meal tickets for the foreseeable future! No doubt the sheer complexity of the storage was also sold as a security/encryption feature too.
Isn't that how all government databases work?
... so how about taking the initial Batch Files, replacing the data of everyone who shouldn't be in it with a null record and then uploading them onto a new system before purging the old system entirely?
can we assume
If the figures of 1 million samples illegally held is correct then nearly 5 quid a pop to look up a record and delete is somewhat costly.
I cant see what batch uploads have to do with it, if they have to compare samples they dont get results back in blocks of 1000!
Sounds like idiot Gov muppet getting the shaft from IT person that can spot idiot a mile off.
Re: can we assume
So they have to remove 1 million records? So paying minimum wage to someone manually deleting entries would actually work out cheaper than this quote, assuming they can handle the neck breaking speed of 2 an hour?
While we're helpfully throwing SQL about
TRUNCATE TABLE [DNA]
By the way, what did you mean by 'just the innocent ones?'
I'll charge them half that
After all petrol isn't cheap these days.
The IT person missed an opportunity here..
When we knew that we were explaining technical details to someone who had no idea what we were talking about we'd throw in Star Trek references. 'The deflector dish is misaligned' - 'The dilithium crystals are spent.' - We never got picked up on them.
Jiggery pokey going on
Who did they hire to delete the data at that cost , the BOFH ?
Can't be the BOFH.
He'd be storing them to null to speed up the loading scripts.
Not fit for purpose
If this is the case, then the DNA DB is not fit for purpose and there should be serious questions asked of the supplier of the system.
Did they just build it thinking that our civil liberties would never ever be restored after more than 10 years of erosion (and bankruptcy) by Labour?
Please tell me it's a normalised relational database and there's some sort of common ID field linking the DNA profile with a human. If it is, then the hard bit would actually be trying to get together all the ID numbers of the innocent humans in the first place. The actual delete should be a simple process after that.
Surely under the data protection act people can just go and inspect this data and request for it to be deleted? Oh I forgot, the law doesn't apply to governments and their gravy train agencies.
RE: Fit for Purpose
Given that the purpose was to collect DNA data from everyone who came within swabs-length of the UK and to hold that data indefinitely then the database actually was fit for purpose.
It's the twats who decided we needed a database of this nature in the first place who were not fit for purpose.
I've worked on Police and Courts IT systems in the past.
Last time was in about 2007....I had to upgrade their access 97 database that they were using to run their court reporting software...
Banning slavery was expensive too
Let's not do the right thing if it might cost some money.
Missing the point ..
... only the most god-awful fucked up DB should make it that expensive to delete data, even if you have to wipe all the backup tapes and create new ones.
DNA records stored on an iffy database?
If it's really that much work to remove records (and I don't doubt it is), surely the efficiency of the whole databse can be called into question?
Does rather sounds like an explanation worthy of a spooty youth in Dixons or Comet, mind you.
"Well, yeah, it'll cost, uh, four point eight million because we have to, uh, load them up in batches of a thousand at a time."
"But these are ones we want to get rid of."
"Yeah, but, well, then the ones we've loaded up won't be in thousands no more, so we have to load them up again. And that needs Barry from Head Office, cos he's, like, better at this."
And so on.
While we're on the subject of databases...
... the root of the problem can of course be illustrated by executing the following SQL commands:
SELECT * FROM [civil_servants] WHERE [location] LIKE 'whitehall' AND [clue] IS NOT NULL;
(0 rows returned)
SELECT * FROM [civil_servants] WHERE [location] LIKE 'whitehall' AND [self_serving] = TRUE;
(server crash - result set too large)
Any banker out there want to take a lead...
and donate/recycle some of their bonus to a worthy cause.....
The plaudits they would get would be worth the small cost to them, surely?
Bet they 'accidentally' forget to delete these records from the backups
you think they keep backups ....
with a system as incompetently designed as this one, it wouldn't surprise me to learn there are no backups ....
How much does it cost to do a DNA profile?
I can't imagine it's cheap, private companies charge about $250 for paternity tests etc. So assuming that $1=£1 and add a 1000% government contractor markup they are paying a few quid to put innocent people's DNA into the system in the first place
If you waited for them to be convicted before doing the profile than you could save the cost of removing the innocent ones pretty quickly
Seriously? Did the civil servant/politician who received this estimate just blindly accept it, and not do any kind of mental figuring or apply any sort of common sense to ponder just how ludicrous it is? The fact it comes a day after a huge government announcement about how they're going to do things different now, and no longer accept getting repeatedly screwed by the big IT suppliers just makes me despair. The kind of money quoted would pay a team of 20 consultants £750-a-day for OVER A YEAR. There is no way that even a government IT contractor could figure out a way to string out a few SQL deletes that long. Anyway, they seem to know there's a million innocent people on it... rerun that query with DELETE instead of SELECT COUNT(*) and the job is good as done.
So the sooner they get started the fewer they will have to remove.
Let me guess they *still* have not implemented the rules on *not* putting people on in the first place.
But watch ouf for the old "National security" clause
The impact assessment is here.
It states that "Evidence" from police and security services state national security investigations could last 25 Years (no doubt without *any* convictions). But they wouldn't mind if such a case was reviewed every 3 years or so.
And yes it does think that there are about 1 million people wrongly on the DB so that is c£5 each.
For what I have *no* idea. Perhaps the National Police Improvement Authority (who run the NDNAD) could explain the intricacies to someone from Vulture Central.
It's a very *grudging* start. 3 years is *still* too long IMHO for some not *charged* or convicted.
It would cost a lot more if they did the job properly
Since there will be many innocent people still on the database - those accused of but not found guilty of a "serious offence" - the cost is not a true reflection of how much should really be spent.
Remember, Not Guilty does not mean "Got Away With It" or "Probably Guilty".
As I have said before, this is nothing but bullshit PR and will remain so until *all* records of innocent people are deleted.