Flash data storage: Knocking TAPE off the archiving top spot

Maybe

I could see low-cost, write-rarely flash taking over from disk as an archive medium if it was developed.

Currently, flash is being developed to give high access speeds, high density etc. If a mfr decided to, I could see "slow flash" being produced at much lower costs. Possibly more bits per cell, much slower reads with error correction, and probably a full wipe to modify data. The controller wouldn't necessarily need as many channels, it could access a string of flash ICs over a single channel, addressing one at a time.

It would take a manufacturer who wanted to do this, though.

Doesn't matter, it goes something like this...

"A table of hardware costs Wikibon has prepared shows that, looking at 10-year cumulative hardware costs, a disk-only archive costs $5.5m while tape-only is far lower at $0.8m."

Shock face.

There's a Dilbert cartoon moment out there somewhere.

"Tape isn't sexy, let's kill tape"

"What's sexy got to do with it, it is a lot cheaper"

"I read in an airline magazine that facebook has a cool archive, we need to do something similar or we won't be cool"

"We don't have budget"

"I'll find the money, I think there are several projects that are a bit fat as it is"

"What will it gain us in the long run?"

"I'll call up me buds and we too will be in an airline magazine, free adverts, plus we will be cool!"

Hmm...

"The key to this approach will be surfacing metadata, currently buried on tape cartridges, to a flash layer that can signal the location of desired data on the tape. Combined with linear tape file system technology, we believe this approach will deliver better business value for the right use cases."

- Methinks Quantum can already do all of that with their StorNext FS and Storage Manager.

Consider LTFS, but add -LE

When you have LTFS-LE (Library Edition), where a single translation layer presents a single hierarchy, where top-level directories are individual volumes, you can actually use tape as a drop-in solution.

Multiple use-cases, multiple architectures

There are 3 drivers for long-term retention of digital content

1) I want to reuse, repurpose, and re-license (eg movie studio)

2) I want to analyze (eg Amazon and Facebook)

3) I want to preserve because that's what I do (US-LOC) or because I promised (Shutterfly)

Each brings different sensitivities in terms of performance SLA, cost, and data integrity. For use-cases 1 & 3, tape-based latencies are entirely acceptable as long as sequential performance is good enough for bulk data operations. Analytics will almost always need more consistent SLAs.

In all cases however, the placement, maintenance, and performance expectation for *metadata* is much more aggressive than the SLA and placement rules for asset itself. For these sorts of storage solutions, the query has always dominated as the first step in data IO, although to-date most often performed against a host database. In the future, storage solutions optimized for these use-cases should recognize that semantic through distributed indexing mechanisms implemented in flash. And although the LTFS file-system may be is useful at a component level, it is inadequate for the task at an aggregate solution level.