Forcing digital forensics to obey 'one size fits all' crime lab standard is 'stupid and expensive' Opposition is growing over demands that digital forensics labs comply with ISO 17025 – an international checklist for laboratory testing. Essentially, the UK government and Brit police chiefs want computer forensics labs serving Blighty's criminal justice system to be ISO 17025 compliant by October 2017. That means IT experts …
Maybe the computer forensics industry should create a standard that makes sense. The problem with an ISO that does not make sense is that even in civil litigation, non-compliance will be used by opposing council to undermine the results. So everyone in the biz will eventually have to adopt it if they want to stay in business. The only way to fix the situation is for the appropriate professional organization to create a sensical standard that, over time, can become the officially required standard. Not having a standard is not an option.
>but digital forensics has no professional body
Step 0: Join the BCS if not already a member. (I would question the integrity of any digital forensics person claiming to be a 'professional' if they weren't already a member of a recognised IT industry professional body)
Step 1: Form a BCS special interest group.Join the Cybercrime Forensics Specialist Group
Step 2: Get the group to use the BCS professional standards umbrella to define a set of good practises.
Step 3: Use this platform to get the standard adopted internationally.
Yes the above will involve people's time, however if there aren't people/companies with deep pockets who are prepared to fund a new fully independent professional body it is probably the fastest route to gaining credibility, picking up a ready made set of basic professional standards, getting established and gaining access to relevant funding agencies.
> "Maybe the computer forensics industry should create a standard that makes sense"
Digital Forensic specialist here. We do indeed have such standards. In the UK, they are known as the ACPO Guidelines. They are specific enough to set a general SOP, flexible enough to offer exceptions for the many edge-cases that exist and updated regularly enough to incorporate new advances.
What about
ISO/IEC 27037:2012 Information technology -- Security techniques -- Guidelines for identification, collection, acquisition and preservation of digital evidence
"provides guidelines for specific activities in the handling of digital evidence, which are identification, collection, acquisition and preservation of potential digital evidence that can be of evidential value"
It would be lovely if the standards authorities would listen to the industry - public and private sector - before compiling and applying the standards but there does not seem to be much sign of that. Consequence - a lack of understanding by the regulators means nonsensical demands placed on the industry and a collapse of the sector as plummeting forensics budgets meet escalating cost burdens as DF becomes a loss generating occupation.
The ISO committees do listen to industry. We've all been consulted recently on the proposed changes to ISO/IEC 17025.
As to whether labs should use the standards or not, that's down to the demands of the customer (whether private or government). If they want an accredited lab to do the analysis, that's their prerogative.
The second pie chart showing that 70% of companies have only spent up to £10K to become compliant could make people believe that compliance is fairly cheap. Instead, it could be that these companies have become compliant as it was cheap to do.
What would be really useful is a chart showing how much companies, who are not compliant, think it will cost them to become compliant.
"The second pie chart showing that 70% of companies have only spent up to £10K to become compliant"
Are we reading different articles? The second pie chart in this article shows that 70.8% of people have no idea how much it cost. The two slices representing under £10k aren't labelled, but look to be under 5% in total. Given that by far the largest portion that actually gives a value is the >£50k one, I doubt anyone is going to be misled into thinking it's particularly cheap.
I ran a lab of 12 forensic analysts doing mainly criminal analysis on behalf of various police forces. 17025 would have cost us around £75k to become compliant, with an on-going annual cost of around £15k to remain certified. Given the wafer thin margins that public sector digital forensics operates on, these costs are prohibitively high and unsustainable.
Otherwise known as doing it properly with records i.e. ISO9001 without the old manufacturing bias. Minimum (external) cost is about £2k to £3K for the first audit for a small lab, then £1K per year after that, plus your own time of course. Any labs not already certified to ISO9001 may find this hard, but the 2 standards are very closely linked and compliance with ISO17025 means you are operating in accordance with ISO9001 (says so in the introduction).
I worked for ISO9000 software companies.
ISO9000 seemed to be based on the idea that good paperwork is proof of a good product. Sure, when you delve into a disaster project the paperwork is poor or non-existent. But bad software design will not be fixed by meetings that are scrupulously minuted, with actions duly chased up.
Many a good piece of software has come from a flowchart on the back of an enveloope.
All the costs are set out on UKAS website.
UKAS have been changing to make life easier for the smaller business. In the old days, we had a one-day visit by two assessors each year, every year. Whether it was a full reassessment (every four years) or just a surveillance visit (the years in-between).
Now this past year, as it was surveillance year we just got the one assessor on a one-day visit going over our processes and documents.
Result - accreditation costs more or less halved. And should just point out that if you are working to the standard and the assessor doesn't identify any "improvement actions" that have to be followed up, then that cuts the costs of assessment too.
As a small bonus, there is no yearly fee for being accredited.
ISO9001 is a much less complicated and far lower bar to clamber over.
17025 is orders of magnitude more complex to prepare for and more difficult to maintain.
And given the fact that it's simply not an appropriate standard for DF work in the first place, are you surprised it's not wanted?
Conspiracy theorists are probably frothing at the mouth with the thought that the government is forcing a standard on digital forensics that is hard to meet in the hope that the digital forensics can never be used against them in future court cases.
Great idea that; it would also hamper the ability of the police and security services to prosecute anyone who used any sort of digital system in planning or executing a terrorist offence. The restriction would work both ways, you know!
Doh!
"The restriction would work both ways, you know!"
No, a simple law change in the interests of 'thinking of the children' and ' winning the war against terrorism' would very easily resolve that problem - just word it so that evidence against the government, MPs and their mates has to be of the highest possible standard and then nobody who matters will be affected, will they?
I was going to use the joke icon, but some Sir Humphrey is probably already working on the draft legislation.
I am a volunteer technical assessor for a national accreditation authority (not UKAS) specializing in one of the primary sciences, and have done forensic work over a number of years, including as an "expert witness". The ISO 9000 series demonstrates that the paperwork is in order and that you have a working Quality System - It does not accredit technical competence. You could be manufacturing widgets, and follow ISO 9000 and still make a crap product, you would be able to eventually improve the product by following up customer complaints, but it could take a very long time. Unless you can demonstrate competence, you should not get ISO 17025 accreditation. I cannot see how you could justify a procedure that does not accredit competence in an area as important as this, as all of your competitors would have to also be accredited; but it would certainly drive small companies and the "lone academic" out of the industry.
AC
The technical competence is the bit that makes ISO/IEC 17025 different from 9001. In fact its there in the title "General requirements for the competence of testing and calibration laboratories"
It's not enough to have a functioning document system, audits etc but you have to show you know what you are doing.
And not only does the idea of complaints driving up quality fit well with idea of delivering a service for determining the guilt or innocence of the accused but the ISO 17025 standard requires complaints, and mistakes to be noted, acted upon and monitored for trends.
A company that couldn't maintain adherence in that area would lose accreditation and the cops would stop using them. Because that's another element of 17025, instead of assessing the company for competence itself, the customer can point to the 17025 accreditation and say they used someone who knows what's what.
The arguments are about the suitability of ISO 17025 for digital forensics. The standard makes sense and is likely to be financially viable for traditional forensics labs that specialise in sets of single tests - is there a match or what level of confidence is there in a match? Digital forensics deals with PCs, smartphones etc which are whole scenes of crime and where there may be many different files and artifacts all of which require separate testing for reliability - and be tested by validated/verified means.
But there are other avenues open to the courts to test technical and expert evidence, including following the Criminal Procedure Rules (CPR 19), the requirements of disclosure under the Criminal Procedure and Investigations Acts and, where appropriate pre-trial meetings between experts under CPR 19.6.
The article implies that it's only commercial labs that are unhappy with the 17025 standard, but this isn't the case. I have yet to find a police lab analyst who supports this implementation. Most police labs (the ones i have visited) that have had this thrust upon them are struggling to do any work as more and more time is taken with following procedures that were taken from DNA analysis and poorly implemented as a new digital forensic one. Consider that 17025 specifies that exhibits cannot be outside the exhibit store for more than n minutes at a time. This is because DNA samples must be kept refrigerated, this is simply not appropriate when analysing an iPhone.
And please understand that almost every single forensic analyst working in the UK today strongly supports the idea of having a standard, it's simply the fact that 17025 is an inappropriate standard for this particular discipline.
"Consider that 17025 specifies that exhibits cannot be outside the exhibit store for more than n minutes at a time."
Not true - that is not in the standard. The word "exhibit" doesn't even exist in the version I have. It may be in your procedures, but that is a different problem.
Specifically from EN ISO/IEC 17025:2005
"5.8.1 The laboratory shall have procedures for the transportation, receipt, handling, protection, storage, retention and/or disposal of test and/or calibration items, including all provisions necessary to protect the integrity of the test or calibration item, and to protect the interests of the laboratory and customer."
I'll paraphrase the remaining three paragraphs in the section on looking after samples.
5.8.2 says lab needs a identification system for samples
5.8.3 is about recording if the sample is not in an expected state when received
5.8.4 says that handling procedures need to be in place to avoid sample deterioration, and that eg if fridges are needed temperatures etc need to be monitored. That the sample should be kept secure if necessary.
ISO 17025 is regarded as both inappropriate – even useless – and expensive
Therefore, it's like most government-mandated standards. Of course they will force it on everyone! It's just government being government.
(Full disclosure: I work for a government entity)
I've worked in a lab working towards 17025 accreditation and I remember working in that same lab before we had considered it. I can say that seeking accreditation used up the vast majority of our training budget and that our productivity dropped substantially because of other factors.
I can say from personal experience it resulted in no quality improvements. I would go so far as to say there were times when the 'approved' method was in reality bad practice in digital forensics.
I can also say from personal experience the loss of time and resources meant having to do a less in-depth examination than we used to do. For me 17025 will result in a drop in quality.
I really fear for the future of the field I've dedicated my career to over a very long period of time.
I like many people will leave police work if I find the standard prevents me from doing the sort of quality of work I am happy to put my name to.
There are two other commercial analytical (but not computer forensic) labs that do what we do.
I'm fairly sure the other two labs have more resources, people than we do.
But I can say, no matter what other noises the others make about quality, that we are the best lab in the UK for the work we do. Because I have an accreditation to 17025 and the analysis we do is specified on that accreditation schedule.it shows we are 1) competent 2) consistent 3) make corrections when we are wrong.
.
Hi,
About three years ago I was asked to act as an independent expert in a criminal matter in which an iPad had been examined by a private contractor and where their evidence was already part heard in court. I was asked to review their evidence and noted they had achieved ISO17025 accreditation.
It was one of the worst pieces of work I had ever seen, with a generally poor understanding of how the device worked. They had failed to record information that was pertinent to the case and had made assumptions which reflected a real lack of understanding of what the device can do and how it might have been used.
There are numerous incidents already where ISO accreditation completely failed to ensure the correct result was obtained and they will continue to happen.
I stand by my words when I say this accreditation offers no guarantees of quality.
I also stand by what I said when I participated in the first open consultation, it absolutely will result in money being diverted away from training, equipment and staff numbers. In fact I've witnessed that personally.
I strongly believe that in 5-10 years time ISO17025 will have been abandoned and will be seen as a very expensive failure.
oh wait they did that already ( ACPO Good Practice Guide for Digital Evidence (Published March 2012)) and theres ISO27037 with 27041, 27042, 27043 or 27050 that seems to better fit the bill.
As for a proffesional body, take your pick IACIS, ISFCE, DFCSC, CSoFS, (ISC)2, GIAC
Hi,
17025 was picked by the 'group of specialists' of which the FSR speaks because these other ISO standards weren't available to them at the time and they didn't know they were coming.
When I highlighted the emergence of these other standards to a member of that group, they looked at me blankly having not heard of them.
As for ACPO, the steering group who have been pursuing 17025 have previously said in the minutes of their meetings they wanted use of the ACPO standard to be discontinued.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
