back to article No hack needed: Anonymisation beaten with a dash of SQL

Governments should not release anonymised data that refers to individuals, because re-identification is inevitable. That's the conclusion from Melbourne University's Dr Chris Culnane, Dr Benjamin Rubinstein and Dr Vanessa Teague, who have shown that the Medicare data the Australian government briefly published last year can be …

  1. ThatOne Silver badge
    Big Brother

    The solution is simple

    Make a law prohibiting people from even mentioning the problem. It's cheap and allows you to make money selling peoples' information as you see fit. If someone complains, fine him. If he insists, put him in jail. That will teach him not to interfere with your god-given right to make a quick buck.

    (Sarcasm? I'd wish. Unadulterated reality, actually...)

    1. This post has been deleted by its author

    2. Anonymous Coward
      Anonymous Coward

      Re: The solution is simple

      That's genius you've just also solved the problem of the Streisand effect, if the law says you can't mention it then it stops spreading because everyone will obey the law and nobody knows.

      (I too studied sarcasm)

    3. caffeine addict

      Re: The solution is simple

      I think you've just invented the super-injunction.

      You'll be rich! Rich! As long as no-one stops the UK press talking about it.

  2. ecofeco Silver badge

    This was proven years ago

    See title.

    1. Phil Kingston

      Re: This was proven years ago

      This is Australia - always a bit behind.

      1. Bronek Kozicki
        Coat

        Re: This was proven years ago

        ... and at the bottom.

        Yes, I know the way out.

  3. Anonymous Coward
    Anonymous Coward

    'Make a law'

    Because outside of the immediate jurisdiction where the law applies, every single data broker will be laughing their asses off, followed by every single hacker / cyber crim anywhere.

    1. Doctor Syntax Silver badge

      Re: 'Make a law'

      "outside of the immediate jurisdiction where the law applies"

      Except, of course for the US which believes its law applies everywhere.

    2. Anonymous Coward
      Anonymous Coward

      Re: 'Make a law'

      This is why the EU's GDPR applies to pretty much everyone in the planet, whether or not they're a european company. You can choose not to play by our rules, where we've defined pseudoanonymised data to be effectively PII, but you'll be choosing to play elsewhere or face a hefty fine.

      It becomes a lot less of a laughing matter at that point.

      1. Charles 9

        Re: 'Make a law'

        Unless they're trying to enforce it against a company with virtually no physical European presence with which to enforce their sovereign power...

  4. Anonymous Coward
    Anonymous Coward

    It's one thing to make a law...

    But enforcing it is a whole different ballgame.

    For a moment I thought that it can't get any worse, are our politicians this stupid? Then today I learned that the modern generation of todays time, at least in Holland, can hardly write. And no: with writing I don't mean jamming on a keyboard, I mean with pen and paper: actually write. You know: the thing you can do even when the power runs out? Not being dependent on...

    So then I read this article again and suddenly it made more sense: as time passes by people tend to get more stupid every cycle.

    1. Dan 55 Silver badge

      Re: It's one thing to make a law...

      Wouldn't that mean cavemen were all geniuses?

      1. VinceH
        Trollface

        Re: It's one thing to make a law...

        Well my step-father isn't, and he's a caveman.

      2. Kiwi
        Boffin

        Re: It's one thing to make a law...

        Wouldn't that mean cavemen were all geniuses?

        Given the lack of "sum total of human knowldge" at that point, and that they managed to even survive against the elements and the environment, I'd say they were pretty smart!

        The information one has available does not show one's intellect, it's how one creates new information or fills in the gaps in their knowledge that shows.

        With the knowledge resources you have today, if placed in their environment, could you feed, clothe and shelter yourself? Could you make tools to hunt with? Could you make a stone blade to help cut fibre?

        And even if you can, how many university-educated people today could manage it?

        I'm not so sure their intellect was as limited as many make out. Total knowledge sure, but intellect?

        1. ImmortanJoe

          Re: It's one thing to make a law...

          I could, yes. And so could most people I know. What would happen to all the people out there with no independent thinking skills is a scary thought. It would probably become quite violent and ugly in short order.

          Back on topic. Trying to paper the cracks with legislature is disgusting. As an Australian, I really wish we had an actual say in how our government is run. To date about the most powerful tool we have is change.org because it can draw enough attention to an issue that if we are very lucky the people involved will decide it will look worse for their career if they ignore it than assert their authority and demonstrate their power over the people.

          Remember the whole medicinal marijuana thing recently? They agreed under immense pressure to put it through and ensured that the law was functionally useless when passed.

          All we have going on is yet another clumsy attempt to assert authority by people that couldn't look after a pet rock.

          1. LaeMing
            Unhappy

            Re: It's one thing to make a law...

            And we (as a group) keep voting them back in! Says a lot about us (as a group), doesn't it!

    2. Anonymous Coward
      Anonymous Coward

      Re: It's one thing to make a law...

      So then I read this article again and suddenly it made more sense: as time passes by people tend to get more stupid every cycle.

      Just watch the film 'Idiocracy' - it is now becoming prophetic.

    3. MrDamage Silver badge

      Re: It's one thing to make a law...

      > "are our politicians this stupid?"

      The simple answer is yes.

      The more troubling answer is "yes, and they think they know what's best for us."

  5. eldakka

    This reminds me of an anecdote about Richard Feynman when he was working on the Manhattan project.

    it goes something like:

    To relieve his boredom and assuage his curiosity, Feynman taught himself to pick locks, open safes, and so on. A lot of it was social engineering rather than pure 'safecracking' like, for example, using common dates or numbers/algorithms - e.g. some scientists used numbers based on 'e' for example.

    So he'd go around Los Alamos picking locks, cracking safes and so on.

    In response to this, a Colonel (I think it was) in charge banned Feynman from entering his offices - rather than fixing the safes to make them harder to crack.

    This is exactly what the government is trying to do with their new legislation.

    1. Steve Davies 3 Silver badge

      re: Richard Feynman

      Have an upvote for mentioning him. "Surely your're joking Mr Feynman" is well worth reading.

      1. Adam 52 Silver badge

        Re: re: Richard Feynman

        May I also point you in the direction of James Gleick's "Genius: Richard Feynman and Modern Physics"

      2. handleoclast

        Re: re: Richard Feynman

        "Surely your're joking Mr Feynman" is well worth reading.

        As I recall (almost certainly incorrectly) it was that autobiography (could have been the other one, maybe even both, and probably elsewhere, too) where he asked which direction a rotating lawn sprinkler (the type with the S-shaped arms) would rotate if you put it underwater and pressurized the water (relative to the air pressure on the sprinkler's hose. He made a convincing argument that it would go one way. Then made a convincing argument it would go the other way.

        He didn't answer that one in the book (or books). Sorta like Eric Morecambe's running gag about the two old men in deckchairs and one says "It's nice out., isn't it?"

        ---------Spoiler Alert------------

        --------------------------------------------

        "It's nice out, isn't it?" "Put it away before you get arrested."

        The sprinkler doesn't move in either direction. But if you're careless about the experiment, you end up flooding the basement where the tank was.

  6. Anonymous Coward
    Anonymous Coward

    "criminalise unauthorised research into re-identification"

    Does that mean there is authorised research? That would imply that some will have authorisation to re-identify what is very personal information.

    Here's a better idea, before you release it pass it through trusted university researchers so they can show you it can't be truly anonymous then don't release it.

    Makes you wonder about all that data google already has over here. I'm sure they haven't attempted to re-identify it...

    1. T. F. M. Reader

      @AC: Here's a better idea, before you release it pass it through trusted university researchers so they can show you it can't be truly anonymous then don't release it.

      This may be a better idea, but it is still not good enough: "trusted university researchers" can only show they cannot re-identify the data, not that it can't be done.

      And that's before we ask, "trusted" by whom?

      1. Anonymous Coward
        Anonymous Coward

        Good point.

        I had another idea, put the data online and let the public enter search parameters to find themselves to show them how easy it is to re-identify, then prompt them to write to their MP.

        1. Lysenko

          I had another idea, put the data online and let the public enter search parameters to find themselves to show them how easy it is to re-identify, then prompt them to write to their MP.

          How about putting the complete medical, financial and educational records of all MPs online, after running it through the officially approved anonymisation process. If no-one has managed to reverse that back to individual identities inside maybe 6 months then maybe it's safe to try it out on the general public?

          1. Anonymous Coward
            Anonymous Coward

            "If no-one has managed to reverse that back to individual identities inside maybe 6 months then maybe it's safe to try it out on the general public?"

            I imagine most of the Cabinet will be outed within hours, if not minutes.

    2. Dan 55 Silver badge

      Does that mean there is authorised research?

      There can be... There just never is.

      1. Primus Secundus Tertius

        @Dan55

        The national anti-spy agencies would disagree with that assertion.

    3. Tom 7

      trusted university researchers

      These will be like lawyers etc working for the government on large contracts - they very rapidly become extremely well paid working for the contractors to ensure there is no-one on the public side that can point out the errors of their ways.

  7. Anonymous Coward
    Anonymous Coward

    No shit Sherlock.

    A database full of fields that can be sorted, full of PII that's supposedly anonymized. Pick a field, sort it to show all the other records with that same PII. Oh look, you've just narrowed your potential suspects list from millions to perhaps a few thousand. Enough such sorting cycles on other fields, finding matching PII in all of them, further reduces the likelyhood of a false posative. Eventually you're left with nothing but records with all matching PII, which means you've just found all the records, supposedly anonymous records, pertaining to said subject. It might take you or I a few days to do such massive computational gruntwork, it will take a government (or government aided) supercomputer cluster a few minutes at most to do the same job. Thus your pseudo annonymous lie is exposed.

    Using Rot13 as your "encryption" doesn't do you any favors either.

    Fuckers.

  8. Ken Moorhouse Silver badge

    How about the use of Cascading Temporal Surrogate Keys?

    Keys are generated at intervals through the day. Researchers only have access to Views that use these Keys rather than the underlying ones. Time-limit queries so that any kind of useful Cartesian Join will time-out. By the time the researcher asks for further data it will not be possible to join it to data previously collected.

    1. LeoP

      Re: How about the use of Cascading Temporal Surrogate Keys?

      Ken, while I very much like the technical idea behind your proposal (and upvoted accordingly), it does run afoul of Leo's 1st theorem:

      The realms of stupidity, lazyness and incompetence do not overlap with realms that house technical solutions. They do however overlap with realms containing educational solutions.

      (Please forgive me for a possibly bad translation, non-native speaker of English)

    2. Charles 9

      Re: How about the use of Cascading Temporal Surrogate Keys?

      If you can read it, you can copy it, by wetware ot the analog hole of necessary.

      No, it seems the only practical solution is to not have the records at all, since they're basically proving something distinct enough to be individual can ALWAYS (part and parcel) be identified.

    3. Lysenko

      Re: How about the use of Cascading Temporal Surrogate Keys?

      Screen scrape the data and then store it offline keyed on a hash of the non-key fields. You then have a stable PK for each logical tuple and can track key collisions in a counter field (assuming some denormalization in the source).

      The entire concept is flawed. If you have enough attribute data regarding an entity then it will always be possible to resolve the PK in any quasi-normalised dataset because the corollary of a tuple being dependent on "the key, the whole key and nothing but the key" is that the key is dependent on the attributes and for any reasonable dataset, some of those attributes will have high cardinality.

  9. John Robson Silver badge

    Just stop...

    Don't try to anonymise the data...

    It's clearly too hard a problem for those who are trying to do it - and if it isn't anonymised at all then people might realise that it isn't anonymised....

  10. Primus Secundus Tertius

    Aggregate data only

    E.g. mean and standard deviation for postal area SW1, but not for Mr/Mrs X of SW1A 0AA.

  11. Anonymous Coward
    Anonymous Coward

    Infectious deseases and privacy

    The story of a new hospital where the doctors were to be housed in open plan offices was shared with me recently. The head of infectious diseases wasn't happy with the potential compromise in patient safety so he rang the politician and charge and said I won't be held responsible if the information about a cabinet member was to leak because of an overheard phone call. Within days the department was relocated to a private section.

  12. Winkypop Silver badge
    Facepalm

    Brandis

    Recently expected to become High Commissioner to London.

    We already have Numpty-in-chief Joe Hockey as the ambassador in Washington.

    Such a shallow talent pool.

    Such plum appointments.

  13. Dr.Strangelove

    Ignorance and hubris simply displayed...

    What this illustrates with blinding clarity is the total lack of even basic knowledge of matters digital, internet and data driven by the AU attorney general and the entire parliament for that matter. The law that should be drafted is the one that prohibits any minister making a statement regarding anything digital without first running it by some year 12 high school graduates and pale skinned gamers for an idiocy test. Fortunately it appears said AG is to be retired and sent to the UK as our High Commisioner, where I’m sure he will advise all within range on how to protect precious British data during Brexit from those obnoxious Euros over a nice cup of tea...

    I think the neddys at the Dept of Chips & Digits might have gotten the Adams SEPF generator working again, if only for a moment.

    1. Bob Dole (tm)

      Re: Ignorance and hubris simply displayed...

      What this illustrates with blinding clarity is the total lack of even basic knowledge of matters digital

      I disagree.

      What this illustrates is a complete lack of desire in actually solving the problem. It usually requires the ones making the decisions to acknowledge there is a problem that needs to be solved. If your government actually gave a crap about it then it would be solved.

  14. John H Woods Silver badge

    I'm surprised...

    .... that anonymised medical records contain an exact birth date... surely it could be fuzzed by 5 to 10% of the individual's age without hugely impairing research findings using this data?

    1. ThatOne Silver badge
      Devil

      Re: I'm surprised...

      > .... that anonymised medical records contain an exact birth date...

      But how would you de-anonymize it later for resale without the convenient individual birth date?

  15. chuckm

    Mr. Metadata strikes again. What a complete boofhead. Our glorious PM is packing him off to London next year, you Brits can have him.

  16. Anonymous Coward
    Anonymous Coward

    In the NHS

    We employ staff who constantly push against releasing this sort of bulk "anonymised" data, however they are drowned out by some clinicians and researchers who simply claim it'll help health research and that's it - done job to get it released.

    We have Information Governance staff, but we're not really following their guidance on matters like this.

  17. anonymous boring coward Silver badge

    How very unexpected...

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like