Optical archival system - where to buy from?

This topic was created by Ioan .

  1. Anonymous Coward
    Anonymous Coward

    Optical archival system - where to buy from?

    Dear colleagues,

    I have a Drobo with quite some harddrives in it and, as time passes by, I feel more and more pressed to archive on different types of storage than magneto-resistive.

    I found in this section a very interesting post about a sony optical archival device:

    http://www.sony.co.uk/pro/product/itmassstorearchive/ods-d55u/features

    Now. Where can I buy this or something similar? Online at best of course.

    Many thanks for your answers.

    Ioan.

    1. Don Jefe

      Re: Optical archival system - where to buy from?

      Sony? Nooooo!

      In all seriousness, why are you dead set on moving away from tried and true archive solutions? You know it is going to be expensive ('sales inquiry' link), uses Sony file management that's likely to be buggy, unsupported and never be updated and it is simply an oddball choice to trust with your archives. You're likely to have to build a relationship with a new media vendor and nobody I know stocks it.

      I mean, rock on man, be the guinea pig for the rest of us, but archiving isn't something I fiddle with, even with clients who just throw money at me.

      1. Henry Wertz 1 Gold badge

        Re: Optical archival system - where to buy from?

        Seconded. Don't buy Sony. They (not archival specifically, but all product lines I've come across...) tend to be pricey; hardware tends to be nice IF you don't hit any problems that make it unusable. There's the rub, I have encountered bugs ranging from minor to "makes the product useless" on almost all products I've come across. This is of course not unique to Sony.

        What IS fairly unique to Sony is unwilingness to FIX these bugs even when it's easy, their "fix" is to buy the next model. I have two examples.

        1) My friend got a Sony DVD player which, after about 6 months, failed to play certain newer DVDs. This was not an optical fault but some firmware problem. Despite the drive saying ON THE BOX that it was firmware updateable, Sony released *0* updates for it, he called and they flat out told him to just buy the next model (which was the same model he had with the firmware update they failed to release for his.)

        2) Second example, a Sony notebook that began shutting down. This turned out to be a chipset flaw making it falsely trigger thermal shutdown, and it took a year or two in the field for this problem to develop. Fair enough, not Sony's fault it was Intel's. BUT, it turned out to be a *2 byte* BIOS patch to work around this, and every vendor BUT SONY released a BIOS update. Sony said "too bad, buy a new one."

        These are by no means isolated incidents. They ship DVD drives that are LiteOns, but Sony writes their own firmware, actually making them buggy compared to if they just stuck a Sony ID string into stock LiteOn firmware. Newer Vaios have some "off the shelf" parts but with Sony-proprietary firmware for no discerniable reason (which based on reviews make it buggier than stock.) I guess the big exception is Playstation, I can't fault them on the PS3.

      2. Fatman
        FAIL

        Re: Sony? Nooooo!

        Cannot be screamed loud enough!!!!!

        SONY!!!! The purveyors of computer damaging rootkits??

        WHY would ANYONE trust their data archiving to kit made by a company known for being smug assholes.

        You are begging for trouble, if not vendor or format lock-in.

        Go anywhere else!!!! BUT NOT SONY!!!!

        </rant>

        1. Anonymous Coward
          Anonymous Coward

          Re: Sony? Nooooo!

          ignore the anti-sony rants above; the fact that sony's consumer divisions are useless (or evil) is not a reason to exclude this tech.

          however, there are a number of good reason not to consider the ODS system:

          in general I avoid optical backup solutions; I've never used sony's system, but previous optical backup technologies had very limited shelf lives (relative to tape or spinning platter).

          I also avoid any backup technologies that don't have large consortiums of major players backing them. if in 5 years the drive dies will you be able to easily source another?

          as others have mentioned you should really look at tape, and specifically LTO; LTO-6 can hold over 2TB per tape (though if you're looking to save money, going with the immediately previous generation of LTO typically results in much cheaper costs per gigabyte, and the tapes will work in whatever you upgrade too in a few years time).

          I've also a fan of RDX, which is basically a removable hard drive interface; the cartridges are made by half a dozen manufacturers and due to the hd interface, the drives are compatible with whatever larger cartridges are released in the future. that said, it really isn't economical as a backup solution (the cartridges cost 3-5x per GB versus 3.5" HDs). I only mention it here because it seems that sony's system is being pushed at video production, an application whose needs tend to differ from backup, and we currently use RDX in this role. it allows rapid expansion of disk space for those that generate excessive amounts of data (which we prohibit from our main backup systems since it would multiply our backup requirements by 10x).

    2. Kevin McMurtrie Silver badge

      Re: Optical archival system - where to buy from?

      Data on spinning rust can be safe forever. I still have my files from a 1989 hard disk, and even a few from a 1983 5.25 floppy. Keep feeding the Drobo new drives to replace failing drives. Keep a second copy to protect against Drobo failures and human errors.

    3. ragge

      Re: Optical archival system - where to buy from?

      That is some kind of 12 disk Blue Ray cartridges. No one knows how long they will actually hold the data.

      Go for LTO instead. Even LTO-6 is half the price for both the drive and the tapes, and then you get 2,5 TB tapes (+ compression). If you want to save a bit on the drive you can go for LTO-4 or 5, you can later upgrade to a LTO-6 drive, or perhaps 7 when that comes out. LTO drives can normally read and write tapes one generation back, and read two generations.

      LTO tapes are supposed to keep data for at least 30 years. I have seen bugs with the small RFID memory though that has made the drive think a tape is empty though it isn't. At least that should be solvable by masking or removing the RFID tag, but I haven't had the possibility to try that yet (the tape got rewritten before we discovered that). But always keep at least two copies of your valuable data, preferable more, and preferable in different locations of course. The Sony thing seems to have RFID memory two, so that is not necessarily a con for LTO, it could possibly be just as bad with the Sony thing.

  2. Anonymous Coward
    Pint

    Uhm..

    "Now. Where can I buy this or something similar?"

    No idea about something similar; but concerning this device: have you tried clicking the "Find your nearest dealer" option yet?

    Have to say that the question kind of puzzles me, considering that the solution seems to be very obvious.

    And if you need to find more online prices (or stores) then why not ask Google? You know, like this.

    No offense intended here, but it really looks to me as if your main question is a bit obvious.

    1. wilber

      Re: Uhm..

      I totally agree. The first thing I did was click the given link and then the 'Find...dealer' tab and was presented with 15.

      Either loan has poor web-page skills, or is looking for suggestions for a backup to the Drobo and didn't want ask directly, or loan is marketing the Sony product.

  3. Caff

    Tapes

    Tried and trusted go with LTO 6, though its about 3k for the drive and 100 for a 6tb tape

    1. Anonymous Coward
      Anonymous Coward

      Re: Tapes

      Yep half the price of the Sony and higher density storage. The OP did want to get away from magnetic media though. Optical is not going to be cheap and with virtually no standard in disc formats.

      1. Martin Gregorie

        Re: Tapes

        I wouldn't trust optical disks for archival storage at all.

        The reason? All computer 'burnt' CDs and DVDs are using the laser to change the state of a dye and dyes fade with time. I've had computer written CDs become unreadable in as little as 5 years and there's no self-healing capability or even much error recovery available. I'd far rather keep my archives on spinning rust where I can use ZFS or one of the redundant RAID formats to get error recovery.

        Last but not least, I keep hearing rumours that optical drives are likely to become obsolete thanks to their relatively small capacities compared with hard drives or tape. If this becomes true and drives become unobtainium or suddenly the recording formats change and, if you don't do something, your archives will become as inaccessible as, say, NASA's tape archives which can't be read (drives no longer available) or your old data on DS/DD 5.25" floppies (same reason). At least if the archives are on spinning rust you can easily and quickly copy them to the latest drive type.

        My current archival method is to keep everything I care about online but backed by two offline generations of USB drives in a fire safe. This way I'll always have two copies of everything (even during a backup) and at least one copy is always offline where a mains spike/lightning strike etc. can't fry it along with everything else that happens to be online at the time. And I can always copy the archive to a new disk.

        HTH.

        1. the spectacularly refined chap

          Re: Tapes

          All computer 'burnt' CDs and DVDs are using the laser to change the state of a dye and dyes fade with time.

          Firstly, this isn't even true - even consumer level rewritable media uses phase change effects rather than organic dyes. Secondly, don't confuse enterprise grade gear with consumer level tat. They're whole different worlds. Even DVD-RAM is only half a step up but is enough to quote a 30 year media life, and unlike the "archival grade" DVD±R's they can actually back up those claims.

          1. Don Jefe

            Re: Tapes

            I've got scads of 'burned' discs that worked during the first post burn test but don't work now. All are less than 10 years old and all are enterprise grade media. I used to use optical media to backup images but the high failure rate and multiple redundant backups had me move the whole company back to tape. Optical media isn't nearly as stable as it is made out to be. It really isn't even cost effective either. Optical media had its place but it was a transition technology that the world is better off without.

            1. CliveM

              Re: Tapes

              Why do I get the feeling you are talking about some CD or DVD-R's? No one speaks of "burning" the true enterprise media - the very term is an implicit reference to the record-in-one go use pattern necessitated by the crappy sectoring built in to the consumer formats.

              You simply "copy" or "write" to the real enterprise formats just like you would any other storage. you wouldn't need to test them post "burn" either - they're generally tested automatically as part of the writing process. In that respect they area whole lot more reliable than magnetic media which basically takes it on trust that the write worked.

              1. Don Jefe

                Re: Tapes

                If you're too busy validate your archives are successfully copied the then you should speak to your boss about some assistance. And the comments I was replying to we're about CD/DVD, did you not read them?

                1. the spectacularly refined chap

                  Re: Tapes

                  And the comments I was replying to we're about CD/DVD, did you not read them?

                  You also refer to "enterprise grade media". In my book (and I presume CliveM's too) that precludes CD/DVD. Switching between different brands and so on is simply fiddling at the margins: they are nothing but toys compared to the kind of system under discussion here.

  4. Alien Doctor 1.1

    I've got several 1994-era Zip drives if you want them ;)

    1. Don Jefe
      Happy

      Ahh ZipDisks, the high capacity, reusable and affordable format of the future. I've got a few external drives and disks laying about, right next to my MiniDisc players (home, in dash single unit and changer). I'm saving all that stuff for the future retro fad: MiniDisc just provides a purer sound (or whatever justification the hipsters of the future trot out).

      1. Danny 14

        ahh zipdisks. clickety click (whirr), clickety click (whirr). Whaddyamean my disk is dead?

        1. Anonymous Coward
          Anonymous Coward

          Followed by the eject mechanism hurling springs and metal covers across the room...

  5. Anonymous Coward
    Anonymous Coward

    Sony ODS-D55U

    http://www.creativevideo.co.uk/index.php?t=product/sony_ods-d55u

  6. Suricou Raven

    A little googling shows the per-gig cost of even the largest capacity cartridges are far above hard drives, even excluding the drive. So far above, you could afford to double up the drives for redundancy and still come out ahead.

    I fail to see what advantages this offers over archiving to disk or tape. What's wrong with magnetic storage? It works, proven history, widely-supported and usually very reliable.

    1. Danny 14

      Maybe the op has a tesla coil fetish or degausses everything just for fun. There are some strange people on them thar intarwebs.

  7. Anonymous Coward
    Anonymous Coward

    well, if they're like cds/dvds

    Then make sure that you don't label them with a paper/glue label because when it shrinks, it'll warp your disks. Oh, and don't forget to make two copies.

    The last time I did this (1993), i used ink and paper.

    1. the spectacularly refined chap
      Devil

      Re: well, if they're like cds/dvds

      Then make sure that you don't label them with a paper/glue label because when it shrinks, it'll warp your disks. Oh, and don't forget to make two copies.

      These are cartridge load systems - the label isn't going to do jack shit. What is it with people here that equate consumer equipment designed to be a cheap as possible with professional equipment designed for reliability.

      30 years ago I used to load games on my Sinclair Spectrum using audio cassettes. I'd regularly get tape loading errors. Ergo magnetic tape is shit. All magnetic tape so don't start talking to me about LTO or whatever, because they all fail in exactly the way and exactly the same kind of rates.

      The argument is a complete nonsense, of course, but it is precisely the one being made by too many commentards here.

  8. karlp

    While I understand the longevity concerns of Magnetic, the reality is that you have a much better chance of reading your LTO drives in 15 years than your niche optical drives.

    I am aware of the sony product, and it is a good product. However I wouldn't recommend it too you. You really need to transfer your archive footage onto a new medium every 10~15 years anyways.

    Use LTO and call it a day.

  9. CAPS LOCK

    AWS Glacier...

    ... looks good to me: http://aws.amazon.com/glacier/

    Not what the OP wants to hear of course.

  10. J. Cook Silver badge
    Boffin

    Another one for LTO tapes...

    I recently had the opportunity to see if some ten year old archives written to LTO1 could be read when I was running a content indexing /audit on a large number of tapes that we had stored off-site. despite have to locate a drive that could read the tape and tracking down software to do the same, I was able to read the indexes from those tapes just fine, and could have performed data restores if needed. I didn't bother testing the DAT tapes we had in the same project, I already know they work just fine. (The DLTs we would have had to source a drive from, along with the older QIC style tapes. While I've got a drive for the travan tapes we had in there, I'd have to cobble together something from the same boneyard of old machines I have. But again, I have no doubts that they could be read.)

    As I'm fond of saying: how much is your data really worth to trust to something new, from a single vendor with a track record of dropping support for products, and with an unproven track record?

  11. F. Svenson

    Optical is dead, dead, dead.

    Way back, in the mists of time immemorial, magneto-optical was used for long-term archival storage. Many regulated companies used devices like the IBM 3995 optical library (with Sony drives). The appeal was:

    1. An allegedly infinite lifetime for the media.

    2. Random access VS linear access for tape (faster seek times).

    3. Solid state/No power required/shelf stable.

    Unfortunately, MO suffered from some major flaws.

    1. Horrible storage density. (last generation I worked on was 2.6Gb media).

    2. Single-sided media and drives (robots that have to flip over disks are more mechanically complex).

    3. Very high cost for media and drives.

    4. Drive failure rates.

    5. Media failure rates (according to Sony, due to excessive mounts).

    6. Very low performance (If I remember correctly in the 1-10MB/s range).

    Evidently, the market decided that the cons outweighed the pros, because all of the 3995's I ever supported were replaced by LTO WORM and NetApp SnapLock.

    For personal long-term archival storage, nothing beats tape. Old LTO3/4 drives are ubiquitous and readily available. Media is cheap, and it is relatively fast and very dense.

    Fuji estimates that tape media can last 30 years. Since this is likely longer than the interface medium (SCSI, SAS, FC) will be available it would seem adequate.

    ( http://www.fujifilmusa.com/shared/bin/LTO_Data_Tape_Seminar_2012.pdf )

    The industry standard/common sense method to insure against individual tape failure would is to write multiple tapes with the same data.

    However, I would strongly recommend that you use a timeless format like TAR or CPIO. It'd be a shame to archive your precious home-made cat food recipes and be unable to restore them because Android for Desktops v107 can't run BackupExec.

    Also, with LTO, keep in mind that there are rules for which drives can read/write which gen of media.(write -1 gen, read -2 gen). Since drives are really cheap on ebay, I'd recommend getting an extra drive, seal it in an anti-static bag and put it in the safe-deposit box with your tapes.

    1. smsm

      Re: Optical is dead, dead, dead.

      Seconded. Don't buy Sony. They (not archival specifically, but all product lines I've come across...) tend to be pricey; hardware tends to be nice IF you don't hit any problems that make it unusable. There's the rub, I have encountered bugs ranging from minor to "makes the product useless" on almost all products I've come across. This is of course not unique to Sony.

      What IS fairly unique to Sony is unwilingness to FIX these bugs even when it's easy, their "fix" is to buy the next model. I have two examples.

      1) My friend got a Sony DVD player which, after about 6 months, failed to play certain newer DVDs. This was not an optical fault but some firmware problem. Despite the drive saying ON THE BOX that it was firmware updateable, Sony released *0* updates for it, he called and they flat out told him to just buy the next model (which was the same model he had with the firmware update they failed to release for his.)

      2) Second example, a Sony notebook that began shutting down. This turned out to be a chipset flaw making it falsely trigger thermal shutdown, and it took a year or two in the field for this problem to develop. Fair enough, not Sony's fault it was Intel's. BUT, it turned out to be a *2 byte* BIOS patch to work around this, and every vendor BUT SONY released a BIOS update. Sony said "too bad, buy a new one."

      These are by no means isolated incidents. They ship DVD drives that are LiteOns, but Sony writes their own firmware, actually making them buggy compared to if they just stuck a Sony ID string into stock LiteOn firmware. Newer Vaios have some "off the shelf" parts but with Sony-proprietary firmware for no discerniable reason (which based on reviews make it buggier than stock.) I guess the big exception is Playstation, I can't fault them on the PS3.

  12. F. Svenson

    Oh... and store 'em in a faraday cage.

    Also, since you seem to have a concern with magnetic media, make sure you wrap them in tinfoil and store them in a faraday cage...

    Just FYI -- As a test one day, I ran a Western Digital caviar (perpendicular recording) HDD through a tape degaussing machine. It flolloped and danced around, but 10 minutes later, we could still read the data.

    I wouldn't be too concerned about stray cosmic rays and solar flares and media. Things today are pretty damn robust.

    FInally, the MO disks... are made from glass. They are waaaaaaay more (physically) fragile than tape media.

    And did I mention in my previous post that the MO drive failure rate is extremely high?

    You've got to ask yourself:

    1. Will the media still be in commercial use in 15 years.

    2. Will the drive interface still be around on Ebay? (good luck finding a PCIe floppy card for a QIC-40)

    3. Were there enough manufactured to be able to find them on Ebay?

    You can still find SCSI QIC-02 drives: http://www.ebay.com/itm/Wangtek-5099EG24-QIC-02-BLACK-Used-or-refurb-/181044517526?pt=LH_DefaultDomain_2&hash=item2a27181a96

    Not so lucky on SCSI 8" floppy drives.

  13. Ole Juul

    function or fashion

    It seems to me that the OP is looking for the cool factor. Nice to see the many answers here advocating function.

  14. James Gibbons

    MO Drives

    I used 3.5 inch MO drives through the 80's to mid 90's and they were highly reliable if you used the right suppliers. I had the optical lens fall out of IBM drives from rough handling as one example of a failure rate. Don't remember any media failures.

    The old 3.5 drives are impossible to get now and any current drives may have a similar short production life. In the 90's I moved all the data onto CDs and DVDs for archival purposes.

    I would go with LTO if I had the $ to spend, but a good RAID NAS with USB backup is a better cost effective solution. If you have two physical locations with fast Internet between them, RSYNC or other methods can be used to copy data between NAS units. I do this every night with my company's critical data.

    I do not consider the NAS and USB drives produced by the major disk companies as quality units. They quite often have poor cooling systems and will burn drives up if used 24/7.

  15. TheSicilian
    Facepalm

    OK, so you COULD go with a goofy, proprietary format, supported only by Sony, the worst possible company to do business with, at a cost of $250 per 1.5TB WORM media and without an autoloader, without proper connectivity (USB 3? No thanks. SAS, FC-AL, or iSCSI for me), and that will likely be totally unrecoverable 10 years from now due to lack of hardware.

    But, lasers! It's OK, I'm not trying to give you shit for no reason. I just had a C-level talking to me about our data retention plans, but his favorite buzzword was "cloud". I had to explain to him that a full weekly backup, once you factored in transit time, bandwidth and all of that, would cost about $3000. Per. Week. Not counting restores, which would be cripplingly expensive.

    What we ended up getting was a dual LTO5 autoloader (48 slot, FC-AL), surplussed from the local guys. Total cost of the library, plus backup exec licensing, plus 100 tapes, plus a HBA and cabling and warranties and all that jazz was about $11,000, and it can kick out a 1.5TB tape in 2.5 hours (5 if i I want an immediate, separate full verify). The tapes are nigh-unto-indestructible, and can last for, easily, 20-30 years with proper storage. Hardware will be, at that time, probably a little hard to come by (LTO7 will be the last generation that can read them, and will probably be out in 2018-2020 or so), but should still be available, and cheap. We spend about $300 per month sending all the tapes to Iron Mountain just in case the building really does burn down.

    I know this to be true since all, so far as I am aware, all LTO gear is now produced by IBM, which will still sell you a brand new mainframe to run code written in the late 50s with no modifications. That's support, right there. Just swallow your pride and go tape, you will be glad you did. It's still pretty hard to beat for archival storage bang-for-buck, unless you are Facebook or something, in which case, MOAR DISKS.

  16. gilf

    Proprietary format? No thanks. I had my share of Sony MO, Zip and Jazz, I even have a Bernoulli disk somewhere. Don't know how long they last because no drives can be had.

    Raided storage for online

    JBOD for near line - cycled periodically.

    Online for important stuff

    CD/DVD for low importance

    If you worry about an EMP killing all you magnetic stuff, no worries, you will not have anything to read you optical stuff either.

  17. wnielson

    Paper?

    Technically, this is an optical archival system: http://ollydbg.de/Paperbak/

    500,000 bytes per page, at an average price of $.03/page (including ink/toner), comes out to just under $65/GB. It is expensive and annoying (>2000 pages/GB), but it'll last for 100+ years if stored properly.

    Seems like a joke, but really, if the data is that important to you then this is another option. Plus you don't have to worry about vendor lock-in. More at http://www.codinghorror.com/blog/2009/07/the-paper-data-storage-option.html

  18. cosymart
    Holmes

    Parchment

    If you want to read anything in 1,000 + years time print it to quality paper with quality inks. Think of the manuscripts of old :-) Anything digital/analogue/optical/laser forget it!

    1. Cliff

      Re: Parchment

      Hey, the producers who digitally restored the classic 'The Red Shoes' faced the problem of how do you archive film reliably - and decided the best bet and most likely to be at least compatible with the human brain was to archive this digital print back to 35mm, and store it in a cool dry box.

  19. a_milan

    +1 for LTO

    Grab a LTO4 or 5 drive, and change to every other generation.

    The real question is how much data you want to back up, what backup/restore granularity you need and what software you will use to index it all

    HDD's are simply too big to be practical, difficult to remove with a consistent backup set and a total lottery if they will spin up when you need the data afterwards.

    In this respect, MO is probably the best, but too expensive per GB and availability of a working drive 15 years hence is probably zero.

    So LTO remains the "sweet spot" - reasonable capacity per-tape, tapes can be removed and stored elsewhere, cheap and quick. Question is, can Drobo feed a LTO drive?

  20. A J

    Use the cloud dude!

    Pay for a good cloud storage service and upload it all to the cloud. This will be much cheaper than any other option and it gives the advantage of an off-site backup should catastrophe occur: eg. A break-in and theft of all your computer equipment including your shiny new backup device, or a fire that destroys everything.

    Real 'pros' make off-site backups. LTO will cost you more, be less reliable and is not off-site.

    1. jake Silver badge

      @A J (was: Re: Use the cloud dude!)

      No. Just no.

      Keep it in house. If you're largish, go off-site with an outlying office. If you're smallish, hang a server off your Great Aunt Ruth's home DSL line in Duluth.

      1. Danny 14

        Re: @A J (was: Use the cloud dude!)

        Amazon are a big company, im sure they wont lose your data or have an outage.

        oh. wait.

    2. Paul Crawford Silver badge

      Re: Use the cloud dude!

      Cloud is for sharing (esp with NSA, etc) not for critical backups. Put them off site somehow, but on your own terms (no vendor lock-in, and encrypted, and with a *tested* recovery plan).

      Can't say my experience of low-end tape has been good, but equally not so great with optical disks either. Whatever your media is, you need also to plan and budget for recovering it all and re-writing it on to a new medium every 5-10 years to avoid obsolescence (and media degradation).

      Personally for several TB of storage I would go with a ZFS-based NAS (ideally from someone who will bug-fix it, so not Oracle). First sync it on-site, then move it off site and do any diff backups/additions that way.

  21. jake Silver badge

    Another nod to going with mag tape for this kind of thing.

    If you're actively using the data, go with HDDs, but it's tape all the way for archival stuff. In roughly 40 years of computing, I have never lost anything I had stored on mag tape. I can not say that about any other form of data storage.

    My 17 year old Memorex Telex 5600 ATL[1] is still chugging along quite nicely. And I can still get both parts & tape for the old gal. I doubt you'll be able to say the same for the Sony you are drooling over in the year 2030 ... and I'll also bet my system will still be fully functional, if leaking oil & a trifle bruised & battered around the edges at the ripe old age of 34 ;-)

    [1] I got her at scrap prices after upgrading a data center with a larger unit.

  22. Azzy

    Let's run the numbers...

    The cost of just the media for optical drives is just plain not competitive with harddrives.

    That would be okay if the longevity of the optical disks was better - but reported experiences with optical disks are not consistent with them having greater longevity (not that harddrives have a sterling record, of course, but when not in use and stored carefully, they rarely fail to work when you fire them up again). Even if optical media is only 2x the cost per gb of spinning rust (I suspect it's worse), i wouldn't feel confident enough to keep only one copy. Whether using spinning rust or optical drives, I'd feel like I need at least 2 copies of the backup.

    I think the best solution for you is either tape or high capacity harddrives (maybe configured as RAID array, maybe even in their own second 5 bay NAS that is nominally kept powered down in a secure location)

    Let's assume you're backing up 16 TB of data (a 5 bay drobo in raid 5 with the largest HDDs you could find), and let's be assume your data is essentially incompressible (such as video, which is the only way for any normal person to use that kind of space)

    I was able to find a quote of like $7k for that optical drive, and the media seems to run a whopping $300 for 1.5TB media. That means your 16TB of data would be $3300, plus $7k for the drive, for $10300 for one copy, and $3300 for each additional copy.

    LTO 6 tape seems to run like $80 for 2.5TB. The drives are around $2300, plus 50 for an SAS controller, since you probably don't have one. This would be 7 cartridges for $560, so $2910 plus $560 per extra copy of your data.

    LTO Ultrium 5 runs $30 per 1.5TB cartridge. The drives are around $1700, plus 50 for an SAS controller. This would be 11 cartridges for $330, so $2080+330 per extra copy of your data.

    A diskless 5 bay NAS can be had for as little as $100, but it seems like reputable brands start at $400-500. 4TB drives can be had for like $180. 5 for $900. So 20 TB of raw storage in 5 bay NAS is $1400 - you'd run it raid 5 like before and get $1400 per copy of your 16GB backup

    So:

    Spinning rust: $1400 per backup ($1000 if you trust a cheap NAS to house the drives)

    Spooling rust (LTO6): $2910 for first backup $560 per extra one

    Spooling rust (LTO5 ultrium) $2080 for first backup, $330 for each extra one

    Spinning optical dye: $10300 for first backup, $3300 for each extra one.

    Looking at even these example numbers is, I think, a good exercise to keep these options in perspective.

    1. Danny 14
      Stop

      Re: Let's run the numbers...

      if you are running a 5 bay drobo with 4TB disks in raid 5 you should be taken out back and shot. Archive is the least of your issues.

      1. Paul Crawford Silver badge
        Boffin

        Re: Let's run the numbers...

        If using a NAS then you need to consider what happens when a HDD fails, and often the raid rebuild will cause others to croak (or at least reveal sector errors). So you should:

        1) Use double parity if at all possible (i.e. RAID-6 or similar like RAID-Z2).

        2) Perform regular scrubs (i.e. weekly check where RAID system reads all disks and repairs any sector errors).

        3) Use ZFS please, as it has much better error checking and correction, and it will tell you which files are trashed (which a lot of file system's don't).

        4) Use a server with ECC memory (OK, getting expensive I know...)

        I'm not kidding, see the following list of papers:

        http://arxiv.org/pdf/cs/0701166.pdf

        http://research.cs.wisc.edu/wind/Publications/zfs-corruption-fast10.pdf

        http://indico.cern.ch/getFile.py/access?contribId=3&resId=1&materialId=paper&confId=13797

      2. Anonymous Coward
        Anonymous Coward

        Re: Let's run the numbers...

        hi Danny 14,

        I actually am doing that at the moment: Drobo 5 bay, 4x2TB enterprise class disks in mirrored mode (one redundancy). Why should I be shot? The Drobo is only used for back up, the data is still on the computers/other harddrives.

        1. Danny 14

          Re: Let's run the numbers...

          Absolutely not! Raid 10/01 is fine and you should give yourself a pat on the back for not using raid 5. Raid 5 is not (for those sort of density drives) a wise idea.

  23. Anonymous Coward
    Anonymous Coward

    Any outfit that caters to the professional recording crowd zill be able to help you. The only ones I know in the UK are CVP, but I am sure there are many more.

    Be prepared for a bit of a shock, though : price range is in the neighborhood of 4000 quid and that's excluding VAT, and then you have to start buying media.

    Enjoy,

  24. Anonymous Coward
    Thumb Up

    excellent

    hi guys,

    Thanks a lot!

    No, i'm not a marketing drone (that one brought a smile on my face - that's why I like it at the el Reg, people always on their toes), just an independent guy who got really scared from a wild storm a couple of weeks ago (EMP or something similar happened).

    To answer some of the posts here.

    I will not go Sony (I too had problems with laptop, dvd drives from them - though my PS2 still works fine), you convinced me. I found the dealers too, in UK, but couldn't find any in Germany, you know. Well at least not in the first 10 minutes. I was still interested in finding an alternative. Saving data in a crystal will take a couple of centuries still, right?

    There are two options I like: backing up to tape and establishing a trusted sync point to a second NAS over internet.

    Some tactics where really cool: keep also two copies but one of them offline when the other one online, and also using a faraday cage - I will also try to do that.

    Cloud is not acceptable, because some customers might not like it and I'm also concerned with ownership issues.

    Again, many thanks, I got an idea of what to do thanks to you all. Backing up is not my specialty and I was really scared about EMP - which might not be so bad after all, if I execute a plan according to your suggestions.

  25. Anonymous Coward
    Anonymous Coward

    Ok...

    I've been working in Storage, Archive, Backup and Recovery for about fifteen years. My background is UK financials, but I now work for one of the large Software/Hardware manufacturers.

    I would say that optical is pretty much dead, no-one really uses it any more and the disks - unless you're talking super-expensive magneto-optical enterprise quality systems - only last about 5-10 years, and that's with good handling, in a climate controlled room.

    If you have a lot of data and a lot of money, go for a tape robot, with a couple of the current generation of LTO drives, never use only one drive as you should really write on one and verify on another to make sure readability as sometimes drives will be fussy about tapes they will and won't read. Record the data on one tape and clone it to another, store one tape offsite at an offsite storage company. Chose your software wisely, it's all very good to have a tape system, but you need to have software to manage the data on those tapes. TAR on its will not do for anything above trivial use, archive software is typically expensive.

    As you're currently using Drobo, I'm going to guess that the above isn't a goer, as it's really a low end of midrange solution and a Drobo is a workgroup device.

    One of the currently available CAS (Content Addressed Storage) solutions would be a good, although probably too expensive solution. Then you have the problem of backup/protection of your CAS system and you're back to the initial problem. That said possibly you can replicate to a remote CAS system, this would be an acceptable solution. Again, this won't be cheap.

    A filesystem which checks it's contents' integrity, such as zfs or refs is probably the way forward, however you are stuck with disk and an alterable filesystem, this is not really an archive. Personally this is pretty much what I use, my backup server's datastore is replicated to a disk, which is stored off-site during the week and replicated on Mondays, before being taken offsite again. By offsite, I mean my garage, which is about 600m from my house, so a nice distance away in case of fire/burglary etc. The nice thing about this solution is it's inexpensive and easily upgraded over time.

    You could use a system like the above and replicate over DSL to a friend's house, however bandwidth issues will likely put a stop to this. Also, don't use anything like real-time replication, this is an excellent way to lose everything.

    Reliability and expense are the key issues. Tape, such as LTO, will only read n-1 or n-2 generations, this means you're on an expensive upgrade cycle of tapes and drives every few years, sustainable for an Enterprise, but not a private individual. Tapes do occasionally snap, so you will need to keep two of each tape. They are also not a random access device, so can be slow.

    Optical disks don't last, unless you're paying mega-bucks.

    It would be nice to have something different, but I suspect your best option is good old fashioned spinning rust.

  26. Anonymous Coward
    Anonymous Coward

    Sounds obsolete

    I built a fast 10.5T RAIDZ2 GBe FreeNAS server for a little over £1000 a year ago, and moved all my data off my optical media, because it was so slow, a lot more bulky, unreliable, and inconvenient. If you have two rsync mirrored FreeNAS boxen with regular ZFS snapshots, why even have archived backups?

    If you really need static longer term storage, tape is probably easier to store, provided you keep redundant tape drives with commodity interfaces and use free or open source file encoding.

    1. Anonymous Coward
      Anonymous Coward

      Re: Sounds obsolete

      How do you back it up?

      By which I mean, it's all very well to have two, but where is the other one? 10.5TB is a lot of replication over the sort of links which most people can afford, in the event of a disaster, and if you've got them in the same place, it's just a copy, not a backup. Fire and Burglars are not great respecters of the backup infrastructure.

      Also, I have yet to come across FOSS tape volume managers worth their salt. TAR is great, but you need a database behind it so you know what is on which tape.

  27. samwiseganji

    1,000 Year M-Disc for Permanent Archival

    Get the 1,000 year M-Disc @ www.mdisc.com. They have a blu-ray (25GB) version coming out Oct. 1st.

    1. Martin Gregorie

      Re: 1,000 Year M-Disc for Permanent Archival

      Smoke and mirrors, pal. Smoke and mirrors.

      Blu Ray first appeared in public in 2000, 13 years ago, so lets be generous and say its development took as long and they had a prototype running in 12 months. This means the oldest disk that can be checked for readability was written no more than 25 years ago. If you'll believe some salesdroid extrapolating 25 year's experience by a factor of 40 and getting 1000 years of 'guaranteed' storage, you'll certainly want to buy that bridge across New York's East River off me for cash.

  28. jrhy

    flexible hybrid archiving solution MagnaStor

    The management costs of an optical or tape library make them impractical for most individuals--especially if you want off-site storage. ZFS is cool for local storage, but again, poor integration for off-site redundancy (unless you want to administer that too).

    I worked on a product (http://magnastor.com/solutions/) which I think combines the best of tape, disk, and cloud storage.

    The UPSHOT is for less money as you'd spend on one LTO drive alone, you can get about 40 TB-years of great local storage, great private off-site redundancy (e.g. on Amazon Glacier on another continent), it's easy to manage, familiar hardware, and low up front costs.

    MagnaStor:

    - enables you to use your existing locally attached disks

    - is checksummed, does continuous data protection, and block-level deduplication

    - easy built-in replication to other media, local or remote, and to Amazon S3/Glacier, for low-cost, online off-site redundancy

    - uses client-side encryption, keys never leave your machine, so your data remains private

    - automatically repairs/restores local storage using cloud backup (or other redundancy)

    I do have quite a bias, as one of the developers, but we engineered it so we'd like all the answers to questions we'd have looking for an archival solution.

  29. smsm

    I've been working in Storage, Archive, Backup and Recovery for about fifteen years. My background is UK financials, but I now work for one of the large Software/Hardware manufacturers.

    I would say that optical is pretty much dead, no-one really uses it any more and the disks - unless you're talking super-expensive magneto-optical enterprise quality systems - only last about 5-10 years, and that's with good handling, in a climate controlled room.

    If you have a lot of data and a lot of money, go for a tape robot, with a couple of the current generation of LTO drives, never use only one drive as you should really write on one and verify on another to make sure readability as sometimes drives will be fussy about tapes they will and won't read. Record the data on one tape and clone it to another, store one tape offsite at an offsite storage company. Chose your software wisely, it's all very good to have a tape system, but you need to have software to manage the data on those tapes. TAR on its will not do for anything above trivial use, archive software is typically expensive.

    As you're currently using Drobo, I'm going to guess that the above isn't a goer, as it's really a low end of midrange solution and a Drobo is a workgroup device.

    One of the currently available CAS (Content Addressed Storage) solutions would be a good, although probably too expensive solution. Then you have the problem of backup/protection of your CAS system and you're back to the initial problem. That said possibly you can replicate to a remote CAS system, this would be an acceptable solution. Again, this won't be cheap.

    A filesystem which checks it's contents' integrity, such as zfs or refs is probably the way forward, however you are stuck with disk and an alterable filesystem, this is not really an archive. Personally this is pretty much what I use, my backup server's datastore is replicated to a disk, which is stored off-site during the week and replicated on Mondays, before being taken offsite again. By offsite, I mean my garage, which is about 600m from my house, so a nice distance away in case of fire/burglary etc. The nice thing about this solution is it's inexpensive and easily upgraded over time.

    You could use a system like the above and replicate over DSL to a friend's house, however bandwidth issues will likely put a stop to this. Also, don't use anything like real-time replication, this is an excellent way to lose everything.

    Reliability and expense are the key issues. Tape, such as LTO, will only read n-1 or n-2 generations, this means you're on an expensive upgrade cycle of tapes and drives every few years, sustainable for an Enterprise, but not a private individual. Tapes do occasionally snap, so you will need to keep two of each tape. They are also not a random access device, so can be slow.

    Optical disks don't last, unless you're paying mega-bucks.

    It would be nice to have something different, but I suspect your best option is good old fashioned spinning rust.

    1. JamesTQuirk

      spinning rust ...

      I still have functioning ST506 drives here, (including a full height 5 MEG 5-1/4 MFM or something in a really old desktop 8086), NAS is answer, I feel also, Home based now I run Multible Storage Backups, When I buy I buy 2 USB/NAS, its getting a bit strange, but theres a sub 1, 1, 2, 3, and now 4 TB generation, so I can always go back and rebuild some of it at least, but now in last 43 years have ended up with 3 big plastic tubs of Spinning Rust, in storage, I look at fondly, but with my luck a meteorite will land on them, and I lose DATA !!!!

      Maybe I can get some back from the pre usb/nas, CD/DVD Archives .....

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon