back to article Wanna keep your data for 1,000 YEARS? No? Hard luck, HDS wants you to anyway

HDS' federal subsidiary says it has a Blu-ray optical storage platform for long term data preservation, with 1,000-year M-DISCs in prospect. The kit - we say kit although no products have been announced yet, just this "platform" - is for US federal agencies who need to "preserve and archive mission-critical data indefinitely …

Page:

  1. Pen-y-gors

    1000 years?

    Yep, long term preservation of storage is critical, but I'm not sure if a DVD-like thing that lasts 1000 years is actually going to be much use beyond the lifetime of the Third Reich, that other well-known 1000 year thingy.

    The disk may well survive. Will the equipment to read it still be available? I think not. Will the algorithms to decode it still be known? Remember the Domesday Project.

    And why does the Naval Research Dept need to preserve its data for 1000 years?

    Really, if you want serious long term preservation, print it out on vellum, or chisel it into the stones at Stonehenge! Sad but true...

    1. DropBear
      Trollface

      Re: 1000 years?

      Yeah well the Egyptians tried that, and it sure went well... we'd still be gawking at those hieroglyphs if it weren't for those meddling kids the Rosetta stone...

      1. Vector
        Holmes

        Re: 1000 years?

        "we'd still be gawking at those hieroglyphs..."

        I've often wondered what archeologists several thousand years from now would make of all these shiny silver disks that started to appear in the late twentieth century.

        1. Anonymous Coward
          Anonymous Coward

          Re: 1000 years?

          I'll bet some of you think "cloud" is a real media.

          1. Anonymous Coward
            Anonymous Coward

            Re: 1000 years?

            "I'll bet some of you think "cloud" is a real media."

            The salesman told me it was clouds all the way down!

      2. Captain DaFt

        Re: 1000 years?

        "Yeah well the Egyptians tried that, and it sure went well... we'd still be gawking at those hieroglyphs if it weren't for those meddling kids the Rosetta stone..."

        Considering how determined their conquerors were to eliminate all that 'Heathen' knowledge after ancient Egypt finally fell, I'd say they did a pretty good job of preservation.

        No matter how well you future proof your data, if someone is determined to destroy it, you're going to suffer losses.

        1. Techno Musing

          Re: 1000 years?

          Lots of questions... Why do you migrate data in the first place? Tape-to-tape, disk-to-disk...

          USUALLY! Because one technology or format has become obsolete OR something is wearing out/degrading. The format that comes from the original Compact Disc (CD) is still supported and working today in these new devices. That's over 30 years. What digital technology is still backwards compatible to 1982 (beside a digital clock), both in data & media format, and you can readily/easily buy today?

          If you didn't have to migrate your data because you can still easily read it, use it, have it supported with no special device or service, didn't take up power or cooling, why would you migrate? This is the media I'm talking about.

          You might still buy systems and drives for speed, density, supporting new formats (with support of old formats), etc.

          1. Stoneshop

            Re: 1000 years?

            If you didn't have to migrate your data because you can still easily read it, use it, have it supported with no special device or service, didn't take up power or cooling, why would you migrate? This is the media I'm talking about.

            You forgot one thing: physical storage. Every storage location can only hold so much stuff (and stuff with data on it tends to require qualified storage locations), and if you're running low on space it may well be worth it to migrate if every box of storage media now can hold four times as much data as before.

        2. Anonymous Coward
          Anonymous Coward

          Re: 1000 years?

          Not just others... but proceeding kings in Egypt were keen to destroy past knowledge that made them look bad and others look good, even if it was true and helpful. Remind you of anyone else? :(

          1. Anonymous Coward
            Anonymous Coward

            Re: 1000 years?

            And then there was that ugly little business with Amenhotep IV (aka Akhenaten). After he died, it wasn't his successor but the people at large who poured on the Brain Bleach. It was rather fortunate enough remained to figure out enough of his story to make sense.

    2. Knieriemen

      Re: 1000 years?

      The thing to keep in mind is that with Blu Ray, you won't have to do the periodic (and expensive depending on size) tape migrations.

      Storagebod wrote a great post on the time it takes to do petabyte-scale tape migrations - http://www.theregister.co.uk/2013/03/27/storagebod_petascale_archives/

      1. Yet Another Anonymous coward Silver badge

        Re: 1000 years?

        Avoiding the need to keep up with technology changes by using an optical disk standard is the reason we backup all our data to Laservision

      2. Charles 9

        Re: 1000 years?

        "The thing to keep in mind is that with Blu Ray, you won't have to do the periodic (and expensive depending on size) tape migrations."

        Then what happens when you have an optical disc migration instead? DVD migrated to BluRay, and for archival we'll probably be moving from BluRay to Archival Disc unless something else comes along, and even within Archival Disc there will be several iterations for starters. The vaunted 1TB/side won't be available for a few years yet. Heck, even external hard drive tech like RDX requires periodic migration (RDX claims a 30 year life right now, but can you really believe that number?).

      3. Anonymous Coward
        Anonymous Coward

        Re: 1000 years?

        Please READ that article by StorageBod. He does know what he is talking about.

        He uses the time for TWO drives to migrate the data, one from the other.

        A couple of years ago I managed a real life data migration project, or a petabyte of data, from an older StorageTek library to a new library. Something like a dozen LT03 drives for reading the old data, and four LT05 drives on the writing end. The whole thing took a few days, and went flawlessly.

        You could expect to perform something like that seamlessly if you were simply updating the drives in an existing library.

        1. Stoneshop

          Re: 1000 years?

          Please READ that article by StorageBod. He does know what he is talking about

          Err, migrating a petabyte worth of tape data using only two single drives? I assume that's just for illustration porpoises, otherwise it'd just be daft. Which would then also hold for your statement that he knows what he's talking about, unless you left out a negative there.

          If you have a tape collection that size (1000+ LTO3 cartridges of 3.58 million cat pics each) you don't have just a single tape drive; the very minimum is a two-drive multicartridge loader per technology. The problem that someone is blocked from retrieving data from a tape you're not currently handling also disappears as soon as you have multiple drives, although it will temporarily slow down the migration process.

    3. Primus Secundus Tertius

      Re: 1000 years?

      As for "thousand year thingies", there are several western examples such as England, Iceland, Switzerland. In the orient there are China and Japan.

      Apols to any I have omitted.

    4. Annihilator
      Joke

      Re: 1000 years?

      " Remember the Domesday Project?"

      Nope

      "Really, if you want serious long term preservation, print it out on vellum, or chisel it into the stones at Stonehenge! Sad but true.."

      Or, write down the retreival algorithms and store it with the disc. Probably best to store it on one of the 1000 years discs so it doesn't become corrupt. Thus solving the problem forever.

    5. Techno Musing

      Re: 1000 years?

      Funny you use this analogy. M-Disc is "rock" based material.

      1. illiad

        Re: 1000 years?

        rock??? nah, do the research... 'vitreous carbon' - think carbon nano fibre

        'stone' has a problem, if you bend it, it breaks...

  2. Cipher
    Big Brother

    Lois Lerner...

    ...and ten others at the IRS need to hear about this. Oh, wait, too late right...

  3. Lionel Baden

    It would be nice if humanity could start collecting all its data for preservation.

    next step is to make it available within context to all.

    im thinking from a future standpoint not current.

    it would be an awesome project to be a part of though !!!

  4. tirk
    Coat

    In 3014, future historians open the archive to discover...

    Rick Astley?

    1. ItsNotMe
      Facepalm

      Re: In 3014, future historians open the archive to discover...

      Rick Astley?

      Or Farcebook files...or even worse...all of the narcissistic tripe from Twatter? Oh goody.

  5. Brian Miller

    M-Disk: 42,000 pictures of cats

    From their own website: 21 hours of non-HD video, 120 minutes of HD video, or 42K pictures of your cats.

    So in 1,000 years, the archivists will pop one of these into a drive, and see pictures of cats. And they will wonder what the hell is wrong with us!

    1. DropBear

      Re: M-Disk: 42,000 pictures of cats

      ...their boffins will simply conclude cats were sacred animals in our times.

      1. Pascal Monett Silver badge
        Coat

        What, they're not ?

        1. ItsNotMe
          Devil

          Re: What, they're not ?

          Sure they are. Everyone loves a little Pussy...don't they?

      2. Annihilator

        Re: M-Disk: 42,000 pictures of cats

        Maybe they weren't so sacred in the ancient Egyptian days either then?

    2. Stoneshop

      Re: M-Disk: 42,000 pictures of cats

      or 42K pictures of your cats.

      Elsewhere on their site it says:

      Using estimates with average file sizes, each M-DISC™ DVD:

      Holds up to 8,000 photos

      Holds up to 1,200 songs

      Holds up to 240 minutes of video

      Holds up to 100,000 documents

      So, it apparently takes just over 5 cat pictures to comprise a photo

  6. Salamander

    So in a thousand years, the US Navy will still want to look at combat data from today. That is like the British army still wanting to analyse combat data from the Battle of Hastings.

    Why is it that some people have such amazing optimism?

    1. Charles 9

      Hey, people still look up Sun Tzu, don't they? Historical combat data can have its uses in the broader scheme of things.

  7. cosymart
    Facepalm

    1,000 years

    Load of bollocks. How do you prove it? Just look at the technology and sociological advances made in the last 1,000 years. Look at the equivalent from c1080. The Doomsday Book, this was written in Medieval Latin and was highly abbreviated and included some vernacular native terms without Latin equivalents. So, it was written in ink on velum? and we still cannot read/understand all of it.

    How many words and abbreviations in regular use in various Navies (vernacular native terms seems quite apt :-) ) will be understood in 100 years time let let alone 1,000 years.

    Anyone paying good money for 1,000 year archives needs to invest in quality paper, ink and people who have legible handwriting! Oh, and add a dictionary to the list of things to be transcribed :-)

    1. Techno Musing

      Re: 1,000 years

      If the bits (or pages) are gone, you can't even argue your point. You won't even get to prove it's garbage.

  8. Crisp

    The discs last 1000 years

    But how long do the reader writer machines last for?

    1. Paw Bokenfohr

      Re: The discs last 1000 years

      Doesn't matter with M-DISC, as they can be read in a standard DVD drive, only the writer has to be special.

      In 2047, when the last company making NX-Ray (TM) drives announces they're no longer going to make them (they had backwards compatibility to DVD, BluRay and UV-Ray (TM) you see) then whoever still has a (useless) archive on M-DISC will just read it off and transfer to Data Crystal storage.

    2. Primus Secundus Tertius

      Re: The discs last 1000 years

      Of course a DVD reader built today will not be around in a workable state after 1000 years. But the disk could be analysed and a new reader built.

      Some time back I read an article about an optical reader that could follow the track on an old (pre-vinyl) gramophone record and reproduce the music minus most of the background noise.

    3. Techno Musing

      Re: The discs last 1000 years

      It doesn't matter! With removable media, you can throw the drive away, buy a shiny new one, and still read your data. This has been working with CDs since 1982.

  9. RaidOne

    If these BD discs are like DVD-Rs that I used in the early '00s

    They will last maybe 5 years, then all files will be corrupted.That is a problem with organic media. Pressed DVDs are a lot better, but it seems that the aluminum inside corrodes as well.

    HDDs are not much better, hysteresis will kill the differences between 0s and 1s, but much slower. All my backups moved from DVDs to HDDs.

    I am not sure about SSDs, maybe someone can illuminate me on that. Not that I can afford to backup on SSDs yet.

    1. Flocke Kroes Silver badge

      SSD

      Without power, an SSD will eventually collect more bit flips than the error correction can recover. With power, the firmware can read through all sectors, and re-write the ones that were hard to read. Eventually that re-writing will wear out all the reserve capacity and the device will lose data. The manufacturers can plot a graph of life time verses temperature for high temperatures and extrapolate a guess at the life time for sensible temperatures. Real figures take years to collect, and by that time people have moved on to the next generation of technology.

      Ages ago I read about an archive project that wrote new firmware for their disk drives. The firmware automatically distributed copies of the data across the array and monitored the health of the other drives. As long as people swap worn out drives for new ones, the data should stay in the archive, and upgrading to newer technology can be done one drive at a time. The most obvious points of failure are lack of funding and someone accidently deleting everything. I still have more confidence in that lasting 50 years than 1000 year optical disks.

      1. Anonymous Coward
        Anonymous Coward

        Re: SSD

        "Ages ago I read about an archive project that wrote new firmware for their disk drives. The firmware automatically distributed copies of the data across the array and monitored the health of the other drives. As long as people swap worn out drives for new ones, the data should stay in the archive, and upgrading to newer technology can be done one drive at a time. The most obvious points of failure are lack of funding and someone accidently deleting everything. I still have more confidence in that lasting 50 years than 1000 year optical disks."

        One issue is when new technology comes out, like faster SAS speeds or an entire replacement. Then you just cannot swap drives out but a full-on migration.

        1. Charles 9

          Re: SSD

          "One issue is when new technology comes out, like faster SAS speeds or an entire replacement. Then you just cannot swap drives out but a full-on migration."

          But you can still perform it gradually. The big part is replacing the controller tech with one that can bridge the gap, say one with the new tech built in and the old stuff supported with a module. Then you can change out to the new drives as you swap out the old ones. Once the last old drive is gone, the module can go as well.

    2. Techno Musing

      Re: If these BD discs are like DVD-Rs that I used in the early '00s

      Don't compare "consumer" grade media to archive/enterprise grade media. Huge difference in materials used and longevity.

  10. btrower

    Take a hint from nature

    You are all of you huge backups of a system, parts of which are millions of years old. The tried and true method used to preserve the shark design these past few hundred million years is how our older data is essentially already being preserved as we copy data we care about from one generation of technology to the next.

    Could we design a living thing to carry copies of data of interest into the future? Probably. Meantime, though, we need to ensure that as time marches on we make copies and copies and copies.

    I have stuff on tape and disks from decades ago. It is basically not recoverable and some stuff was actually lost. Going with a specific media is not the answer. The answer is to have an ongoing program of backing up and copying forward and to do that you need, not rarefied exotic storage meant to last a thousand years, but rather a ubiquitous interface connected to the rest of the world. I vote for the Internet. That's where real men put their backups, after all.

    Stuff written in the Reg may last longer on paper if someone bothers to preserve them. However, they would have to bother. This way, with it sitting on their disks, being archived at archive.org and no doubt gems like my comments being copied off by others for posterity and scholarly research, it is both preserved and actually available to use. The reason? It is being copied forward to ever cheaper storage because it is easy to copy a rack of hard-disks compared to preserving stacks of boxes.

    1. Charles 9

      Re: Take a hint from nature

      As I recall, DNA is an inexact kind of thing. Which is why even identical twins don't have identical fingerprints. In any event, while sharks may not have evolved much over a few hundred million years, we'd probably be able to note some incremental steps along the way, meaning the copying process isn't very exact.

      1. phil dude
        Boffin

        Re: Take a hint from nature

        Yes, DNA copying is deliberately a little bit noisy - E.coil 1 error every 10e-5 ,but Humans have the same error rate but better error correction. By now the analogy with tape comes up! How many bits error correction do you need?

        The physics of the universe is a very good way to store information - in other words you can't beat entropy, and chemistry is entropy patterns encoded in electron orbitals.

        A self-replicating knowledge machine would have to be thermodynamically stable in (or around) the final state of your information. That, essentially, is the problem that biology solves. We are all thermodynamically probable molecular machines - made out of atoms that simply have no choice but be atoms.

        Your eye, for example, doesn't have that choice as it needs all the bits! Many of them get replaced every day...think about that when you wake and open your eyes - no "booting now" screen!!

        Petroglyph is the best form of storage, probably a QR code. Just look at some of the 300 million yr old fossils that are still living with the same body plan...

        P.

        1. Techno Musing

          Re: Take a hint from nature

          The last article I read on DNA storage, it cost $35K/MB to write.

          1. Anonymous Coward
            Joke

            Re: Take a hint from nature

            "The last article I read on DNA storage, it cost $35K/MB to write."

            I can store that for you at $35K per year... as long as it's to be stored in human form...

  11. AcrossThePond

    Compatibility across media

    While 1000 years is ridiculous, the idea of storing on media designed to last beyond 50 years makes total sense. BluRay and M-Disc are compatible in the same drives for both read and write. The biggest difference is M-Disc is about half a generation behind standard BluRay discs from a size and cost perspective.

    Beyond 50 years is not out of the question though. In the same BluRay drive that supports M-Disc and 100GB+ BDXL discs, you can still insert and play your very first commercially produced CD. http://abbaoncd.wordpress.com/european-cds/european-cds-1982/

    1. harmjschoonhoven
      Coat

      Re: "you can still insert and play your very first commercially produced CD"

      I remember a story form New Scientist that the very first CDs could not survive transport in the cargobay of an airplane, because the lower airpressure drew bubbles in the CD. Of course the problem was solved in subsequent pressings of CDs.

      My coat is the one with the Hollerith cards in the pocket.

      1. harmjschoonhoven

        Re: Hollerith cards in the pocket

        Just for the sake of the argument, 1 TByte of data can be stored in a block of 30 m x 30 m x 30 m of punched cards. Mothballs not included.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon