back to article Brit firms warned over hidden costs of wiping data squeaky clean before privacy rules hit

Not enough companies understand how to properly delete the data they hold – and need to address this if they are to comply with new data protection rules, privacy and security experts have said. Under incoming UK and European regulations, firms will be required to completely remove all the data they hold on an individual if …

Page:

  1. Anonymous Coward
    Anonymous Coward

    It is just not going to happen

    Having been employed at a huge "digital" customer company to manage a project to do just this and having scoped (most of) the complexities and scale and ballpark costs, I then saw the appetite of the organisation for the huge effort involved and the opposition from embedded "business managers". My conclusion was that it was never going to happen and it hasn't. I moved on and the company shovelled some more sand over their heads.

    1. Doctor Syntax Silver badge

      Re: It is just not going to happen

      "My conclusion was that it was never going to happen and it hasn't."

      It's probably just a matter of accumulating enough fines for them to realise that it's a way to save money.

      1. Anonymous Coward
        Anonymous Coward

        Re: It is just not going to happen

        just a matter of accumulating enough fines for them to realise that it's a way to save money.

        If it costs millions, and the fines are the usual slap-on-the-wrist, it'll be cheaper to pay the fines.

        1. Remy Redert

          Re: It is just not going to happen

          How does 4% of global annual turn over sound? Because that's the kind of money we're talking about for serious infractions.

          Companies will comply with the GDPR or they will go out of business.

          1. Anonymous Coward
            Anonymous Coward

            Re: It is just not going to happen

            that's the kind of money we're talking about for serious infractions

            "Up to" 4% of turnover. Just like "up to" 20Mbit/s ;)

            1. Doctor Syntax Silver badge

              Re: It is just not going to happen

              "Up to" 4% of turnover.

              It's still a much higher starting point for calculations than at present.

          2. Destroy All Monsters Silver badge

            Re: It is just not going to happen

            More like

            Companies will comply with the GDPR and they will go out of business.

            That's a problem. Y2K redux, with more data and even lower density of skilled IT people.

    2. Adam 1

      Re: It is just not going to happen

      Fines will be treated as a cost of business and passed on to customers. Whilst supply and demand curves should see a reduction in demand if price rises, that can in practice take a while to flow through because of inertia and frankly some services would still be valued at the higher price point.

      You see this all the time as currency movements make imports or exports cheaper or dearer. Unless one of the competitors can actually figure out a cost effective way to comply which is cheaper than the fines, the customer will pay the fines. Maybe in the short term some vendors might make a sell at a loss market share ploy giving the best of both worlds (ie, compliance + no price increase), but I wouldn't hang my hat on it lasting.

      1. Doctor Syntax Silver badge

        Re: It is just not going to happen

        "Fines will be treated as a cost of business and passed on to customers."

        Maybe, but it will then have a bearing on competitiveness. Those who are fined for failure to comply will be competing with those who aren't.

        1. Adam 1

          Re: It is just not going to happen

          @drsyntax

          > it will then have a bearing on competitiveness

          Exactly as I wrote:

          > Unless one of the competitors can actually figure out a cost effective way to comply which is cheaper than the fines

          There is no bearing on competitiveness unless someone is able to come up with a more efficient way to comply (or a loophole that means they don't need to).

          Otherwise the cost will either be absorbed by the shareholders or the customers. Maybe some companies might strategically sacrifice shareholders' profit to grow market share but eventually customers will pay. If I sell a service for 50 quid a month and my competitors are similar in price and I have a new regulation that costs 5 quid a month, I can either raise prices to 55, decide to live on 45 paying the 5 out of my own pocket or leave it at 50 and hope I don't get caught. Perversely, the latter will also grow market share from those who do comply. Laws of unintended consequences and all that...

  2. Anonymous Coward
    Anonymous Coward

    GDPR brokers ...

    There will be a gap in the market for GDPR brokers - similar to PPI outfits - who will take no-win no-fee cases, and simply hit big companies with GDPR requests and charges.

    1. Jon B

      Re: GDPR brokers ...

      Is it fines only or compensation also?

  3. Chris Hills

    Ouch

    This is going to require some serious effort to remove individual records from backup tapes.

    1. Anonymous Coward
      Anonymous Coward

      Re: Ouch

      Add some WORM storage than in some countries may be, or have been mandatory for some kind of data (i.e. tax records, including invoices, etc.) and it can become pretty complex - because those supports may be archived somewhere, and deleting single records can be difficult or impossible.

  4. Pen-y-gors

    Backups?

    What does the law say about backups? Is there a requirement to delete information from all backups? How about backups on WORM optical media?

    And what about other requirements - insurance policies that run for 12 months may include an element of liability for years in the future. The policy details really should be kept as long as is necessary.

    This all sounds a bit silly - more legislation drafted by people who don't know what they're talking about. Unusual though, for it to come from Brussels - Westminster is normally the expert at drafting impossible legislation (see the 2011 Sun Rising in the West Act)

    1. Steve K

      Re: Backups?

      Although less of a problem these days, does this apply to paper-based or microfiche records (e.g. as a backup or if there are records that old - like with life insurance or pensions?).

      As to your point on life insurance etc. I suppose in this case you could argue that if you are still living then you are not about to ask your life insurance/pension provider to delete the information it has on you under GDPR anyway!

      1. Pen-y-gors

        Re: Backups?

        @SteveK

        Actually I was thinking of Indemnity insurance, not life assurance. If a business has a 12-month Employers' Liability policy, and during that time negligently exposes workers to e.g. asbestos dust, then when the asbestosis is diagnosed 20 years later the insurance company are still liable. It helps to be able to prove exactly what the policy covered, or even if they were covered.

        1. Anonymous Coward
          Anonymous Coward

          Re: Backups?

          HSE: "Health records, or a copy, should be kept in a suitable form for at least 40 years from the date of last entry because often there is a long period between exposure and onset of ill health"

        2. Rich_G

          Re: Backups?

          According to the ICO a data processor can refuse the right to erase data "to comply with a legal obligation or for the performance of a public interest task or exercise of official authority;" - in the case of insurance policies the FCA (SYSC 9.1.2) policy information must be kept for 5 years.

    2. Anonymous Coward
      Anonymous Coward

      Re: Backups?

      There have got to be exemptions for compliance with legal requirements. I'm wondering how audit trails can be properly maintained.

      1. Anonymous Coward
        Anonymous Coward

        Re: Backups?

        "There have got to be exemptions for compliance with legal requirements"

        There are almost no exemptions under the GDPR. What you've got are six possible justifications for holding personal data, ranging from the very narrow (legislative compliance) to the very broad ("legitimate purposes"). They're all reasonable and easily understood. The more broad the justification the more strong your own justification for using it has to be and the more carefully you have to balance the rights of the person.

        For example. If the law says you hold the data for 7 years, you hold the data for seven years and the individual gets no say. If you decide holding every customer you've ever had in a marketing database to sell at a later date with no record of when or even if consent for that was given is a "legitimate purpose" then I suspect ICO might want to have a word with you.

        1. katrinab Silver badge

          Re: Backups?

          You are required to keep details of sales invoices for 8 years or so [1], but that doesn't mean you can use the data for any purpose other than to calculate your tax liability.

          [1] the actual requirement is 6 years after the deadline for the tax return to which they relate, or 6 years after the relevant tax return was filed if later. If you are a company, and sold something on 1st April 2017, and your company year end is 31st March 2018, the tax return deadline is 31st March 2019 (tax is due 1st January, Companies House deadline is 31st December), and 6 years after that would be 31st March 2025.

    3. Nick Ryan Silver badge

      Re: Backups?

      Backups are an interesting case, and still unresolved from the orginal DPA (1998).

      The latest snake oil GDPR consultant I spoke with (essentially a clueless box ticker tasked with bringing in more consultancy and death-by-powerpoint sessions) seemed to think that when an individual's right to erasure is performed that their data must be removed from all backup media as well. They just couldn't understand that it wasn't a case of just "removing" the details from the tape, and I wasn't even going to waste my time running through the fact that backups are almost always compressed, often encrypted, and the backed up data may be in an application or file system format. In other words, to remove "Joe Bloggs" from your backup data you would have to extract the contents of the backup tape to a system what is able to understand and process all the data formats and structures in place, then to remove/overrite all the data pertaning to "Joe Bloggs" and then to respool the data back to tape. All the while not knackering up whatever arcane data formats and structures are in place.

      Obviously this is technically possible, however how many tapes or tape sets do you have? You'd have to perform this action on all of them where you reasonably suspect that data may be recorded regarding "Joe Bloggs". Once this is done you hope that the backup media are still in a working order.

      The following day you receive another request, this time from "Linda Smith" who also requires that her data is erased... and so on...

      1. Anonymous Coward
        Anonymous Coward

        Re: Backups?

        Both the DPA and GDPR are unequivocal. Backups are an information system, any deletion must include these backups also. This is a huge problem not just for backups but any immutable storage. Data warehousing, hadoopery, all sorts of tech has a huge cost inflicted by this requirement. Dismiss it as "snake oil" at your own risk.

        However your complaint highlights your own noncompliance in another, more fundamental way. If the records were encrypted you'd have the option of deleting the (external) encryption key, thus effectively wiping the records from the tapes. If you've properly protected the personal information you've been entrusted with this aspect is not an issue. The "hidden cost" of all of this is that almost everyone has spent the last twenty years happily ignoring the DPA's requirements and now they've got all of six months to comply with the GDPR because of their own laziness.

        1. Nick Ryan Silver badge

          Re: Backups?

          In my experience the issue around backups has yet to be resolved like this because they are a special case.

          As for deleting an (external) encryption key, are you seriously suggesting that the solution is that every data row relating to each and every identifiable individual in a database has a unique encryption key? While technically possible any system attempting to do this would grind to an immediate effective halt as soon as you tried to do anything in it, particularly when you involve data searches, indexes or reporting. Also, deferring such protection to a unique external key would just mean that you have to manage these keys in the same way, tracking changes to them and deleting them from your backups as well. This is just an arbitrary central and singular database, then there are the ancilliary records and files related to an individual as well as this would mean that you would have to encrypt every document in this way which given that a document could refer to more than one identifiable individual starts to get incredibly messy.

          1. Anonymous Coward
            Anonymous Coward

            Re: Backups?

            "As for deleting an (external) encryption key, are you seriously suggesting that the solution is that every data row relating to each and every identifiable individual in a database has a unique encryption key? "

            Yes. It is commonly (but confusingly) called "tokenisation" and is exactly how banks and government comply with this level of fine-grained requirement.

            And yes, I know exactly how hard a problem it is, it's what I do for a living. The reality is that now with the prospect of real, meaningful fines you can no longer just ignore that this is the law. If you want to have a plethora of high-volume, fine-grained data you either need to be able to rewrite it quickly (i.e. to delete the relevant record) or you need to be able to tombstone that record through either revocation of a tokenisation IV/key or by using some bigtable-esque trickery.

            1. Anonymous Coward
              Anonymous Coward

              Re: Backups?

              Yes. It is commonly (but confusingly) called "tokenisation" and is exactly how banks and government comply with this level of fine-grained requirement.

              The problem is that there is an inherent problem looming: the conflict between mandated retention of information for compliance reasons and privacy. Even if you have a tokenised format that isolates an ID from the actual person (which is FAR from universally the case), you still need some key lifetime mechanism to ensure that data becomes unavailable afterwards and that is a process matter.

              I'm OK with GDPR being pushed in, because companies have a tendency to park any spend if it's not needed, but OTOH I suspect there will still be some serious tweaking required to really make GDPR work as it should. I just hope that won't happen through court cases.

              Oh, apropos privacy consultants - yes, there are many, but few that have a grip on the raw practicalities on making this effectively work, and even fewer know how to make that management efficient. It's truly scary what dares itself sell as a privacy consultant these days, but I guess that's normal for an as yet rather new market.

              1. Anonymous Coward
                Anonymous Coward

                Re: Backups?

                "The problem is that there is an inherent problem looming: the conflict between mandated retention of information for compliance reasons and privacy"

                There is no conflict - the mandate wins.

        2. Anonymous Coward
          Anonymous Coward

          Re: Backups?

          "If the records were encrypted you'd have the option of deleting the (external) encryption key, thus effectively wiping the records from the tapes"

          As long as you do not prove that data cannot be retrieved without the encryption key, now and ever, then you don't comply with GDPR. Throwing the key of the cabinet holding the files is not the same than destroying the files.

    4. jimsneddon

      Re: Backups?

      If it is impossible, or unfeasible to remove individual records from media such as backups without affecting the rest of the data (which in itself could be construed as a breach), then it would be deemed out of scope as far as I know.

      As long as it is documented that this has been considered during the right to erasure process the ICO should be fine with this as far as I understand it (the regulation can be vague in areas).

      Hope this helps.

      1. Remy Redert

        Re: Backups?

        You could probably get away with IF you use incremental back ups and store the deletion, so that any restored database would not have the data you were ordered to delete, even if it might conceivably be recovered from the back ups directly.

        If you're planning to just go "Oh if we restore the back up you'll have to ask for your data to be deleted again", expect to be fined.

    5. jimsneddon

      Re: Backups?

      Also, existing legislation such as keeping financial information under FCA rules will take precedence over GDPR legislation.

    6. RegGuy1 Silver badge

      Westminster is normally the expert at drafting impossible legislation

      Westminster is normally the expert at drafting impossible legislation (see the 2011 Sun Rising in the West Act)

      Or the 'Great Repeal Bill/Act.'

  5. Tezfair
    Stop

    6 years of data

    This got me thinking about backups which hold client data. As I read this, those would also have to be cleaned too, but then I thought about the 6 year data retention required by the gov etc. How can a company clean old data but still be compliant?

    Another way of looking at it is I run sage and have backups from day 1 of my biz. If Mr Johnny down the road comes to me and says I need to remove all his data, that will be impossible without deleting the backups, which the tax inspector will demand I restore as needed. Even editing sage won't be enough to remove him and satisfy any future audits.

    1. Doctor Syntax Silver badge

      Re: 6 years of data

      Short memories. A week ago we had this article: https://www.theregister.co.uk/2017/08/07/data_protection_bill_draft/

      In that there's a link to https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/635900/2017-08-07_DP_Bill_-_Statement_of_Intent.pdf

      In there is a brief description of what's proposed which includes the following qualification: as long as it is no longer necessary or legally required for the grounds on which it was originally collected, or there are no overriding legitimate grounds for processing. (p13 for those who want the entire paragraph).

      That should cover the requirement to retain data for HMRC. It's a basis for at least arguing that the difficulty of removing from backups is an overriding legitimate ground for not doing so but you'd then need to have a plan for removing the data after a restore - retain the script for doing the original removal might be a start. It might be a good idea to look at just how many generations of backup you really need.

      OTOH is you're retaining data about previous customers to sell on or pester them you have my complete lack of sympathy.

      1. Tezfair

        Re: 6 years of data

        I missed the article and thanks for the additional link. Appreciated.

      2. Anonymous Coward
        Anonymous Coward

        Re: 6 years of data

        Catch 22?

        "basis for at least arguing that the difficulty of removing from backups is an overriding legitimate ground for not doing so but you'd then need to have a plan for removing the data after a restore - retain the script for doing the original removal might be a start"

        May require recording whose data you need to remove on a restore... in most circumstances record ids to be removed may be okay. But thinking about this, I've come across another question: is there any requirement to keep a record that a request was carried out?

        1. Doctor Syntax Silver badge

          Re: 6 years of data

          "May require recording whose data you need to remove on a restore."

          If you don't have surrogate keys in your database now might be a good time to think about adding them. A cascading delete of record 0e32b622-814a-11e7-8d87-78acc0c6193c is a bit less personal than one for Fred Bloggs. Even without GDPR requirements its also a much better database key.

  6. Alister

    None of the best practices quoted in the article are of any use in complying with the GDPR's requirement to allow the complete removal of all data relating to an individual, unless that individual's data is all located on a single physical drive, which is highly unlikely.

    It is therefore impossible to follow best practice in order to comply with the directive, and in practical terms it would require multipass overwriting of portions of databases, and sections of backups, without disrupting the integrity of the rest of the data, the technology for which is not readily available at present.

    1. smudge

      it would require multipass overwriting of portions of databases, and sections of backups, without disrupting the integrity of the rest of the data

      In my last job, exactly this problem came up. We were operating a service for a client, and it had a massive customer database. The client said "How do we delete our customers from the system?". Our f**kwit designers said "You don't, because you never asked for that facility." Of course, the f**kwit designers had never thought to ask if it would be necessary, nor realised that compliance with data protection legislation would make it essential.

      So this kicked off a big study, and, yes, one of the findings was that it would be extremely difficult to delete customers without disrupting the integrity of the database. When you have all sorts of links from customers to financial information, to reports, and so on - some of which will identify the customers, but some of which which will merely use customers' data - then you have to be extremely careful not to screw everything up when deleting a customer. No point in complying with data protection legislation if you are now producing false accounting information!

      Of course, it would have been easier if deletion had been designed in from the start. There were also other oddities, such as financial legislation requiring maintenance of customer history for x years (where x varies from country to country). Also the paradox that you might need to keep details of a customer, maked with a flag to say that they don't want to be contacted by marketing.

      Lots of lovely problems to keep everyone occupied!

      1. Michael Strorm Silver badge

        "Also the paradox that you might need to keep details of a customer, marked with a flag to say that they don't want to be contacted by marketing."

        Exactly the problem I'd been thinking of! Somewhat reminiscent of this...

        LISTER: Holly, is there something that you want?

        HOLLY: Well, only if you're not busy. Would you mind erasing some of my memory banks?

        LISTER: What for?

        HOLLY: Well, if you erase all the Agatha Christie novels from my memory bank, I can read 'em again tonight.

        LISTER: How do I do it?

        HOLLY: Just type, "HolMem. Password override. The novels Christie, Agatha." Then press erase.

        LISTER jabs two-fingered on a keyboard.

        LISTER: I've done it.

        HOLLY: Done what?

        LISTER: Erased Agatha Christie.

        HOLLY: Who's she, then?

        LISTER: Holly, you just asked me to erase all Agatha Christie novels from your memory.

        HOLLY: Why should I do that? I've never heard of her.

        LISTER: You've never heard of her because I've just erased her from your smegging memory.

        HOLLY: What'd you do that for?

        LISTER: You asked me to!

        HOLLY: When?

        LISTER: Just now!

        HOLLY: I don't remember this.

        LISTER: Oh, I'm going to bed. This is gonna go on all night.

        1. This post has been deleted by its author

      2. Lotaresco

        ' "How do we delete our customers from the system?". Our f**kwit designers said "You don't, because you never asked for that facility." '

        It's not the designers who were f*ckwits.

        HTH.

        1. smudge

          It's not only the designers who were f*ckwits.

          To be accurate.

          1. Destroy All Monsters Silver badge
            Holmes

            Fuckwittery can be allayed with large amounts of money and project extensions.

            Just saying.

            Some people expect everything is free.

  7. Anonymous Coward
    Joke

    Thinking outside the box!

    I'm keeping all corporate data in OneDrive for Business, so I'm sure a MSFT cock-up will delete it soon enough.

  8. Anonymous Coward
    Pirate

    Hurrah for Brexit

    Once we leave the EU, this silly nonsense can be binned!

    1. JimmyPage Silver badge
      Mushroom

      Re: Once we leave the EU, this silly nonsense can be binned!

      Only if we don't want to do business with the EU.

      1. Dan 55 Silver badge

        Re: Once we leave the EU, this silly nonsense can be binned!

        That's quite possible. The likes of Rees-Mogg think Empire 2.0 would be sullied by merely trading with the EU. That there isn't enough rest-of-world to make up for the amount of trade that the UK does with the EU doesn't matter.

      2. Jamie Jones Silver badge
        Big Brother

        Re: Once we leave the EU, this silly nonsense can be binned!

        I'm sure general human rights, cosumer protection, and general data protection will be head of the queue.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like