I have to admit that I started to think along the lines of fast catalogue recovery in the event of a database error, or a loose tape of the kind that has a coded data tag but no human written label; but then I thought, "Why the heck am I trying to find a justification for this idea?"
Spectralogic CTO talks up hybrid flash-tape cartridge. Welcome, tape robot overlords
The idea of a hybrid flash tape cartridge was floated by SpectraLogic CTO Matt Starr in a conversation yesterday. He said LTFS was becoming more and more popular but LTFS information on a tape couldn’t be read until the LTO cartridge was in a drive. This takes time as, in a tape library, the robot has to transfer the …
-
Thursday 10th December 2015 12:54 GMT Joerg
Tapes will become too expensive.
And unless they are going to use 3D XPoint technology the standard NAND flash SSD drives are just too unreliable and too easy to damage.
If they could add 8GB or 16GB 3D XPoint SSD cache for $2 to $5 at most then it might be useful. Otherwise it would be useless.
And anyway such a system would need triple-checking of all data on the SSD cache to avoid that the metadata got corrupted and data on tape then unaccessible. The system must be able to quickly check and discard any corrupted metadata on the SSD by retrieving the original copy on the tape itself.
-
Thursday 10th December 2015 13:06 GMT Roo
Re: Tapes will become too expensive.
If you *really* want to do this surely having the cache at library level rather than tape level would make a ton more sense because the expensive memory would then be caching the "most in demand" metadata rather than metadata that isn't touched for 5 years before the tape is binned.
-
Thursday 10th December 2015 22:01 GMT Anonymous Coward
Re: Tapes will become too expensive.
Flash is cheap, and 8GB would be way more than you need for metadata information here - you really only need pathname, size and tape location information for each file. No need for triple checking, ECC protection is fine. This is just a cache of metadata that already exists on tape, so if it went away entirely you are no worse off than with a tape today.
A bigger problem is that unpowered flash is not always non-volatile over a scale of years, though that would be easy to work around by having the LTFS driver regularly insert each tape and check/refresh the metadata.
Caching in the library is a bad idea. What happens if you have it replaced? If you want a centralized store of metadata, why not store it servers? Backup servers keep this sort of information today in their catalogs, so I see no reason why a couple servers couldn't keep replicate copies of the cache instead of the library or the tapes.
-
Monday 14th December 2015 20:39 GMT Trevor_Pott
Re: Tapes will become too expensive.
FFS
3D FLASH != 3D XPOINT!
AUUUGH
Listen: 3D flash is a production technology for NAND flash. It is cheaper than TLC which is cheaper than MLC which is cheaper than SLC. 3D Flash good, if you want deep and cheap flash.
3DXPoint is a frankenstorage whathamawhosit that is a completely different technology from 3D NAND flash and will be priced at $holy_shit until such a time as real competition enters the market for its niche. The 3DXPoint niche is "using NAND as a tier of storage between NVMe flash and system DRAM".
In order to accomplish its goal, 3DXPoint will have to have much better write endurance than traditional 3D Flash, which is actually quite shitty. This is part of the reason it will be so expensive. In other words:
3D Flash = Great price, shitty write endurance.
3DXPoint = $Make_the_pain_stop, fantastic write endurance.
There is no rational reason to use 3DXpoint on a gor'ram tape drive drive just to serve as a glorified high capacity RFID tag. No tape gets written to that many times. You want 3D Flash for that job.
AUUUUGH!
-
-
-
Thursday 10th December 2015 23:17 GMT Alan Brown
Flash in cartridge
Is already there in LTO.
It's a RFID chip with a few kB onboard and it's already being used in proprietary ways to provide some LTFS information, but that's more along the lines of being able to look up the tape and content in a database held off-tape(*).
In any case, LTO isn't "run from one end to the other and you're done". The data is laid out in bands so you run to one end of the tape, move the heads a little, then spin back, move the heads a little and move back again (serpentine layout).
Except that's not what happens. The "heads" span the entire width of the tape and the drive activates individual coils within it.
What that means is that you're never usually more than a few tends of GB away from where you want to be. if you're on the wrong track you just switch to the appropriate one, then skip forward/back until you find the spot you need. (Did I mention there's another track on the tape which is there to index the start/stop points of each block of data? LTO is full of underappreciated cleverness and it's a shame the drives are silly-expensive.)
(*) This principle is how Bacula backup system works. It can tell you that MNO version of file XYZ is stored on ABC tape in file 1234, block 345 - and what that means is that when the tape is loaded into the drive it will skip to _exactly_ the point that's needed and haul out the file in a few minutes, instead of the traditional way of loading up ABC tape and spooling along it for a few hours until you find the file you need. Because the database contains checksums of every file it can also be used as an extremely effective IDS.
-
Friday 11th December 2015 13:37 GMT toughluck
Right now, cartridge memory (CM) in an LTO tape is 16 KB. That's how much you need. The tape will read the full inventory in the header on load, which takes only a second or two, and quickly space to the proper point along the tape.
If anything, increasing the capacity of CM to 1 MB would be possible and would solve some 50% of problems with low capacity of the CM (although would introduce another one -- it takes bloody ages to read that CM chip due to low throughputs involved). NFC with Bluetooth could probably solve that throughput problem, at the expense of power required. Adding a second chip for redundancy wouldn't hurt, either. Cost would go up, and that might not be appreciated.
As I see it, though, there's just no need for extra connected memory. If you want to index the tape contents, do it at the host level. That way, you're making the index easily accessible to all applications, you can quickly rebuild the last known good index to tape without having to re-read most of it and you don't have to rely on a mechanical connector that's prone to breaking.
Okay, reductio ad absurdum:
If adding a small amount of memory makes sense, adding more would make even more sense. Why not go the whole hog and replace tape in the cartridge with flash memory, then?
Food for thought: You could fit thousands of micro SD cards in a single 4×5×1 tape cartridge (or hundreds of USB sticks, etc.). Then add a beefy connector at the front (like aggregated 8 USB 3.1 ports) and off you go. With USB 3, you could get away with using the slowest sticks in the "cartridge", have 8×USB 3.1 hubs and putting all sticks in it in a RAID.
I wonder if there's someone making all the decisions, reading this and deciding: Hey, that's actually a good idea, we'll build it.
Biting the hand that feeds IT
