Sign in / up

LTFS and ugly ducklings

This topic was created by Chris Mellor 1 .

COMMENTS

Post your comment
House rules Send corrections
Add to 'My topics' unstar *
  1. Chris Mellor 1

    LTFS and ugly ducklings

    If it quacks like a duck, looks like a duck and swims like a duck then it's a duck, and not a swan. So what is LTFS?

    LTFS is a way of providing file:folder type access to files on tape using drag and drop operations. You are no longer forced to use a backup application or equivalent software to move files to and from tape and so, the story goes, an obstacle to wider tape usage is removed.

    My query is; how much of an obstacle is it? If I, as a user, have file:folder access both to an external disk and to a tape drive; on which device will I store my files? It will be disk, natch, because access is faster and there will be a backup, probably also on disk, in case disk numero uno goes tits up. Tape is still tape, still slow compared to disk, even inside an LTFS wrapper.

    If the company I work for already has a tape system and it implements LTFS then yes, I could use the tape but why would I want to do that? If I was forced too then, fine, reluctantly I'd use the damn thing, but sneak in USB sticks to make life easier where I could.

    There's a rumour that one large tape system-supplying vendor has not one LTFS-using customer in Europe. It wouldn't be surprising. For everyday access to files, putting LTFS access on tape in a disk-using world, is like putting lipstick on a pig. It's still a pig.

    Am I right or am I being a dickhead about this and missing a point or points?

    0 0 Reply
    1. Chris Mellor 1
      Reply Icon

      Re: LTFS and ugly ducklings: LTFS pitfalls

      A vendor sent me these points about LTFS:

      I would like to invite you to examine a list of caveats that ALL LTFS adopters need to pay attention to before they simply abandon whatever "proprietary" software they are currently using before moving all of their eggs into that LTFS basket.

      In truth, this issues with tape and the general storage population were more related to capacity and performance rather than any problems with vendor lock-in. When a user chose a vendor's solution, they generally standardized on that solution - whether a tape technology or a software model - so not being able to read a DTL tae in a VXA drive was not at the heart of any displeasure on the part of the user. Rather more that they needed a week and major automation or staffing investments to create a backup to the existing tape technologies when they could accomplish the same apparent backup to a disk array in hours with no addition staff or education requirements.

      The sad fact is that the tape drive vendors solved the primary issues with the advent of LTO-5 technology. With a proven throughput of 140MB/sec - 200MB/sec and capacities of 1.5TB to 2TB per tape (real numbers, not mythical marketing fluff), the capacity and performance issues became non-existent.

      It was actually the unexpected and undisclosed announcement of IBM at NAB in 2011 (the remaining LTO.ORG members weren't even aware it was happening at the time) that has caused further fracturing in the market space as many existing tape software vendors were improving their tape support and offering much more robust solutions thanks to the combination of capacities and performance of the LTO-5 technology. Now, the LTO.ORG members had just placed a shot across their bows that warned that the work that so many had done for so long was now no longer applicable.

      There are many aspects of tape that LTFS does NOT take into account, however. No verification of data written to the tapes. No mechanism for spanning writes across multiple tape volumes. Serious recovery issues if a reset or power glitch occurred during the writing of data to an LTFS tape. No easy way to track tapes that are not currently mounted on your system.

      And my favorite glitch - there's no single point of support for an LTFS user when things go wrong (and they quite often go VERY wrong). Since it's open source, it's pretty much a case of "you broke, you get to keep all the pieces" when you need help. The response is generally "the source code is freely available..." But, how many small businesses or production companies have staff who are familiar with low-level C/C++ coding at the kernel level with a complete understanding of the low level operation of tape devices? On the other hand, that "openness" can also result in many splinter implementation as users decide that they can do this or that better.

      --------------------------

      I've anonymised the post in case the vendor meant it for me privately - but the points are the points.

      Chris.

      0 0 Reply
  2. sbrume

    Re: LTFS and ugly ducklings: It is not your fathers tape solution

    Tape is not disk, anyone who says different is pulling the wool over your eyes. That does not mean that an open-standard for putting data on tape that allows customers, vendors of software and IT professionals to retain data for a longer period of time without worrying about retaining a DB of file locations, is a bad thing.

    The arguments:

    - Tape is slower than disk - spot on, transactional data is not viable on tape.

    - Tape at a local consumer level is an overkill - absolutely! Take a pro-user with 60TB of videos or picture and ask them to spin the disk.....

    -LTFS has no data verification - Not true. Not only do modern tape drives do read while write verification, but they also support end to end CRC checking that can have block based verification run against it at anytime.

    - If the tape is stored offline there is no way to know where a file is located - True, except LTFS is so easy to develop applications to that it is easy to create a catalog, reference a start up 1Delta ( www.1delta.net ) that has a catalog application that is light weight and works with any filesystem device.

    Unlike traditional ISVs, if the catalog is ever lost or destroyed, a simple mount of the LTFS cartridge reveals every directory and file on the media.

    LTFS is in its infancy, but has already gained traction with vendors and developers creating solutions to meet more modern data movement and access requirements. As with Linux early on the open source does leave some support questions, but major vendors such as IBM and HP offer full support for LTFS and the solutions they have developed around LTFS.

    Is it really a slight to those companies that have been working to improve the tape usage that IBM came forward with a new tape usage/format and then allowed anyone and everyone to use/develop to it for free!?

    It is still tape, if a exception happens during the write of data it does require a recovery of the filesystem, but as with any product there are tools for that.

    If you want massive storage at a proven lowest cost and longer retention/upgrade paths, tape has always been the choice. LTFS allows the data to be easily accessed by an application when the data is in a cold state and rarely accessed but supporting an ISV infrastructure to get data in hours rather than minutes is not a choice.

    It is not your fathers tape usage scenario.

    **Disclaimer: I am an employee of IBM and the views here are my own not the views or position of IBM

    0 0 Reply
  3. josh.krischer

    Tape is not disk or Flash and tractor is not a car but each one can be used in optimum way.

    The tape is still the lowest $/GB, the lowest energy/GB, supports input/output etc.

    LTFS

    Self-describing tape format to address data archive requirements

    Tape is logically divided “lengthwise” into two partitions

    Index partition: File system info, index, metadata (37.5 GB)

    Content partition: Contains the files / content bodies (1425 GB)

    Implementation of specific software that uses this data format to provide a file system interface to data stored on magnetic tape

    Tape format developed by IBM (adapted by LTO consortium)

    Defines the organization of data and meta-data on tape - files stored in hierarchical directory structure.

    This format makes it possible to implement software that presents a standard file-system view (letter) of the data stored in the tape media

    Direct access (drag and drop files) to file content data and file meta-data.

    Interchangeability of tapes between systems

    0 0 Reply

POST COMMENT House rules

Not a member of The Register? Create a new account here.

Forgotten password ?

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon