Is LTO-5 the last hurrah for tape?

I don't think...

It'll be the complete end of tape. Even with de-duplication.

With the mainframes you have colossal VTS libraries which do currently have a small disk cache and then a large back end tape cache. Mainframe tape volumes are generally small in size and also yes high in frequency but with a low retention period for most. And if you look at how HSM has evolved on something like IBM's mainframe, the VTS the way it's been designed is very well suited for the OS.

If you move over to open systems and other platforms this then differs. With de-duplication many saves can be made with a good de-dupe rate. But what if you have a library with say 30 LTO4 drives in it and your tape utilisation regarding space on the tape isn't that great? You've got up to 30 hosts writing at 120Mb/s = 3600Mb/s sustained for potentially hours, but yet say the disk array you need for your virtual tape doesn't require the arm/storage ratio because of the size of data required? You'd need hundreds of arms, even though many arrays are looking to use slow-ish SATA compared to something like 15k rpm FC disk. Then pile on replication of your tape/disk cache and you've got a disk array that's going to have a very intense read/write profile, albeit quite a sequential one.

Yes if you exploit good disk technologies you can replicate your storage and back it up completely separate to the host, but not for all operating systems or all environments. And that sort of disk, software and automation is costly to implement.

Also you have to think of large sized backups with a long retention but a long frequency like yearly multiple terabyte backups. You don't want to keep a 10Tb save on a VTS when it's read profile is near zero and the next iteration next year has changed so much that de-duplication hardly claws you back any savings. You want to farm that sort of thing out to tape, preferrably a tape remotely. A 10Tb save could take up 15-18 Tb maybe within a 13 month cycle. Or about 8 tapes with good compaction.

Virtual tape for certain platforms still needs some work regarding the performance for a library serving multiple hosts against the storage/cache it requires to give the thruput when you look at it to replace large heavily used tape libraries, even if de-duplication is not in-line and is an afterwards-esq process.

I will be keeping a keen eye on how vendors are going to deal with tape regarding it's development or if they thing that storage arrays will become so cheap but yet perfom just as well and networking costs to fall that then maybe.. just maybe tape could be redundant.

WORM

How do you enforce a WORM version of tape?

As for "CERN's Large Hadron Collider project will use tape to store the vast amounts of data it expects to generate" - I get the impression that by the time they start generating data, SSD storage will have progressed far enough for them to use that instead and teacher's will be teaching tape in "History of Computer Science" classes alongside punched cards :-D

Manufacturers

I think you're forgetting that there are other manufacturers who make LTO4, namely Tandberg, TDK and Sony that I know of, so it stands to reason that at least some of these should continue to make LTO5 as well.

Did someone miss the bit

where it says manufacturers aren't signing up to the new format? When it stops being cost effective to sell drives/tapes then they will stop being made full-stop whether your individual shop uses them or not. THAT was the point of the article.

Tape is dead??

I've heard this one before. Any big company running mission critical systems has to off-site multi-genaration backups for disaster reasons. Also many companies are subject to compliance regimes that dictate the retention of data for long periods - often years.

Now you can run de-dup system with remote replication to allow for off-siting of your multi-generational backups without (hopefully) using too much disk space. However, de-dup won't work for archive and log type data (such as DB arvchive logs) as every block tends to be unique. Also, there is a penalty in removing all the redundancy in your backup - multiple, full backups to tape gives you truly independent, multi-generational copies. With de-dup you are absolutely relying on the software and hardware not to screw it up. If it does, then it has the potential to render all generatons of your backups unusable.

There's also the environmental issue. 10PB of tapes will only use power when being read/written to. Even if you can get 10:1 reduction in space requirements (and you'll be lucky) then that 1PB of ((RAID-protected) disk space is going to be chewing up perhaps 20kW with the controllers - if you are dealing with the off-siting issue through de-dup replication, then double that. MAID setups don't work too well as, with de-dup, you tend to be splattering data all over the array and can't really shut any of it down.

Now none of this is to decry using disks where it suits. If you have a small enough business, then a couple of USB-connected portable disk drives might do.

Tape is dead, no really...

I can see tape being dead in small companies, it's dead at my house for instance as I use disk to backup and a removeable drive that lives at work and gets replicated once a week. This is for a TB of data.

At work we've got multiple L5500s and multiple TS3500s (very expanded) in many many sites. We have hundreds if not thousands of PB of data stored in these libraries. To replicate this with disk will require a massive investment in cooling and power, even if the drives intelligengly shut themselfs down when not in use.

As the datacentre is driven to reduce power useage, I would expect tape to have a resurgence, rather than die. The only question is - when does a cartrigde get too large? For those who already dupe their backups for offsiteing to somewhere like Iron Mountain this isn't an issue, but for big shops where offsites are in a tape library at another site, I wouldn't store 1.6TB of data on a single disk so I would expect technologies like inline duplication (backing up the same stream to two tape drives at the same time) to come into common useage as a type of RAIDesque mirroring for tape.

WORM

Was just researching tape drives myself for my company and came across this from HP, maybe helps answer the WORM question..

http://h18000.www1.hp.com/products/quickspecs/13258_na/13258_na.html

The LTO-4 Ultrium and LTO-3 Ultrium tape drives features the ability to archive and store data in a non-rewriteable format that meets the most stringent regulatory guidelines. Using a combination of integrated fail-safe features in the drive firmware, cartridge memory, and tape formatting, the tape drives can archive large amounts of data for periods of up to 30 years in a secure, untampered state. Since all LTO-4 Ultrium and LTO-3 Ultrium tape drives include support for both rewritable and WORM media, IT organizations can now easily integrate a secure, long-term archiving solution into their current data protection strategy. As compared to other technologies that feature support for WORM storage, the tape drives offer the advantages of[..]

DeDupe and Compliance

Anyone care to guess what is going to happen when the first case of compliance data being recovered from a deduped file hits the courts...so far the courts haven't ruled on regulatory data being rebuilt from a deduped file...but I can see the lawyer saying 'Is this an exact copy of the file that was given to you'. Tape dead? I don't think so.