* Posts by wcpreston

12 posts • joined 31 May 2018

Firm fat-fingered G Suite and deleted its data, so it escalated its support ticket to a lawsuit


Re: So...

Can you provide any links on this extra-cost G Suite backup?


Re: What use is Google's statement about backups?

That is not what that google statement says. It says they can restore a deleted USER, because a deleted user goes into a recycle bin type area. This is not a deleted user; it's a deleted account. BIG DIFFERENCE.

This is not a cloud computing problem. This is a "user assuming their SaaS service was backing them up w/o verifying that" problem. I've been preaching this for quite a while. This will now be my test case every time I talk about why you should backup your SaaS data.


Your SaaS vendors aren't backing up your data

Please check your service agreements. You might be surprised. Neither Salesforce, Office 365, or G Suite have what I would consider a backup for your data. I would define that as something I could use to restore all my data when the caca completely hits the rotary oscillator.

Salesforce has KIND OF a backup, but it takes 6-8 weeks to get ahold of it and costs $10K. And it doesn't restore everything.

MS and google has recycle bin type features, but those are all stored w/in your account. They go away if you (or a bad actor) delete your account.

What I don't like is that they are not open about this. Either they should 100% cover you and support it, or they should very plainly state that backups are your responsibility. Right now they do neither.


Re: Under a cloud

It's not in the service agreement for them to do so, so why should they? None of the major SaaS vendors (Salesforce, Microsoft, Google) have backup your data in a way that any backup person would recognize as a backup.

Salesforce has a restore service that generates CSVs of your account. It takes SIX TO EIGHT WEEKS, costs $10K, just to get a bunch of CSVs that you can then use to restore your data. Metadata is gone forever. Even they don't recommend you to use it.

All of MS and Google's documented data protection features leverage the recycle bin or versioning, and thus are not a backup. MS does have a delayed replicated copy of your entire account they can use in case of THEIR disaster, but they do not make it available in a scenario like the one in this article.

Please look at your service agreements. Look for the words backup, restore, recovery, data protection, etc. I think you'll be surprised.


Re: Conflicted who and what to bash

I do agree that Google should have notified them quickly that there was no hope. Google does not have a backup of your entire account. As with other SaaS vendors, backups are your responsibility. Feel free to consult your service agreement.

The glorious uncertainty: Backup world is having a GDPR moment


Re: Backups are GDPR compliant trough retention.

Waiting for stuff to expire from backups is a great response. The problem is too many companies keep backups for years, decades, or even forever.


Re: Not a problem

We brought it up before. It's getting coverage now because the law is now in effect. Such is life with news.

There are actually sections in the GDPR that speak to technical infeasibility and undue burden as a defense against certain requirements of the law. In addition, the need to keep the data for other valid business purposes is also a defense.

As to what you're proposing (restore, delete, backup again) for every single request? The cost is so high that most companies would just pay the fine if the law were to be enforced that stringently. We're talking costs in the tens of millions every single time you get a request. My opinion is that is never going to happen. Not to mention the risk of doing something wrong and doing damage to the company.

The ICO said they will provide guidance on this soon, and I for one am looking forward to it. I'm willing to bet the advice is going to be closer to what Robert Wassall said in the article. The data needs to not be accessible to production systems, not be used for any decisions, etc. To that i would add that a company must commit to deleting it if it ever DOES come out of the backup system via some kind of restore.

My opinion so far.


Re: Snapshots

No, I am NOT suggesting anyone do that with their snapshots. But your suggestion has another issue. The snapshot you're suggesting the company delete has other legitimate purposes. It's very common, for example, to keep all snapshots for 30, 60, 90 days, or even longer.


Re: Not a problem

I'm not sure the filing room analogy works here. In this analogy, the filing room represents the main "production" copy of the data. And yes, you WOULD have to delete it in that case.

But imagine, if you will, you had a scanned JPG of every piece of paper in each drawer, stored in a completely separate system. Continue to imagine that you don't have OCR, so you can't scan the contents of each JPG without physically pulling each one up, reading all the words in it, then moving on to the next image. Now imagine being asked to redact info from those JPGs without the ability to search them. Now you're a bit closer to what we're talking about here.

I agree with your last comment. I look forward to further guidance from the ICO.


Re: Not a problem

I can only speak for myself. My blog was a response to comments I was seeing out there that suggested that GDPR RTBF was absolute and everyone should be able to delete personal data from production AND backup systems. So I suggested that wasn't possible given current technology (nor do I think full RTBF from backups is coming any time soon), and suggested the kind of process you mentioned in your comment. So I feel like I'm trying to clear up FUD, not create it.


I had a person reply to one of my twitter comments that this exact thing happened to him. The company "forgot" him, then a while later he started getting emails from them. Upon investigation, they realized the restored the marketing database to before he was forgotten. They had to institute the kind of process we're talking about.


Re: Not my field of expertise

Well this IS my field of expertise, so I'll chime in. :)

Many backup systems have added (or are in the process of adding) features to delete personal data from the backup -- to a certain extent. For example – depending on how your backup is stored – it is technically feasible to delete all spreadsheets, word processing files, or PDFs with a certain person's identifier in them. But asking that same backup product to delete the person's data from within a file or database – while keeping the rest of that file intact – is venturing into extremely dangerous waters. If that's what we're asking, I'll have to agree with the quote in the article form Linus Chang, "deleting data from a backup is a terrible idea because it risks corrupting the backup, breaking referential integrity, breaking applications that were expecting that data to be present, and importantly, breaking any checksums on the data that would prove that a restore was successful."

That leaves the "delete on restore" option. My opinion is that a RTBF journal/database that stores ONLY the unique identifiers (and no other data) – while it sounds on the face a direct defiance of the RTBF – is the best way to ensure the person "stays forgotten" if there is ever a restore from older backups. It's even possible to have the backup system trigger the "make sure these people stay forgotten" process after a restore.

The RTBF article of the GDPR says you can keep data required to defend against a legal claim. In addition to being used for this "make sure they stay forgotten" process, this database I'm proposing can also be used to prove when someone asked to be forgotten, when they were forgotten, etc, in case of a GDPR claim. In addition, the use I'm proposing is also to protect against a legal claim – that you said you forgot somebody that you ended up restoring from backup. Ergo, I think it should be OK to have a RTBG journal/database. I am not a lawyer and am not giving legal advice on GDPR. I'm just spitballing here.

Biting the hand that feeds IT © 1998–2019