Your next storage will be invisible (for a while) In the last two or three years I've talked a lot about "Flash & Trash." A two-tier storage strategy to best cope with the increasing demand of high IOPS and low latency from primary applications on the one hand, and high capacity, associated with throughput at times, on the other. This kind of need depends on the type of …
"For example, if you are running the IT infrastructure in a university, you'll probably have access to a lot of manpower (in the form of students and researchers)… in this case it's probable that your TCO won't be affected by the cost of Sysadmins"
In that case you might be paying for more Sysadmins to defend it all from the students and researchers.
Mine's an old white one with ferric chloride stains that won't come out.
>"cope with the increasing demand of high IOPS and low latency from primary applications"
Thrashing storage has been a problem with applications software for at least 30 years. Its actually an applications design issue but early PC applications writers had no idea that they were supposed to manage storage use, the just use computing resources as if they're infinite and whine when the app doesn't run fast enough.
This is a problem that hasn't been addressed while computing resources expand as fast as resource hogging (with the result that we now seem to need what was a decent sized supercomputer just to read email) but we are going to collide with limits sooner rather than later. We won't be able to address these issues overnight -- they're too baked in -- but future winners will be from the ranks of those that understand resource management and understand that bigger/faster isn't necessarily better.
"...but we are going to collide with limits sooner rather than later."
What kind of hard limits do you think we'll hit given that rust capacity has managed to continue climbing in spite of scares while solid-state capacity is still growing and still has several big shifts left in the tank?
Rebuild time is the name on that steel wall at the end of your drive. Even clusters have to pay attention although less frequently. But when they hit... This comes down to addressing serious process engineering as part of the TCO assessment.
BTW, repurposing boxes this way is something I've been doing since forever. Especially when I wore the uniform. I do use newish hard drives, as per Enrico, with older SSD's for caching.
Flash-based SSDs are interesting as they are fundamentally built of a large number of very small "disks" (pages) that the on-board firmware already retires as it fails.
Thus a "dead" SSD isn't all dead, and in theory at least, could keep being used as pages fail, by stepping down its apparent size.
The hard part is working out when to give up of course - down to 70% or 50% original capacity? Further?
Now, gradual flash chip failure is actually pretty easy to detect and then negotiate (lock the drive to read-only, copy what you can to a new unit, use recovery tools for the rest if needed). But IINM Flash SSDs also suffer from a higher-than-normal rate of controller failures, and controller failures are sudden catastrophic failures: fine one moment, hard-bricked the next, so these need to be taken into consideration as well.
Very well spotted on "location" value there. Economic rent, or rental in the case of data centers, is the biggest factor most often ignored by ISP's and operators using locations. The same enterprise ignorance happens for the tenant trader for any good or service in the high street too. That is, all increases in productivity from the enterprise above the capital and operational costs, always, get passed on in higher rent, in the end. So the only business worth getting into is real estate. And let the so called entrepreneurs do all the work for you for as long as they can remain above water. Notice how the property owner is rarely the data center operator or ISP/IXP. An IXP only survives because everything is donated. The best operator will own the freehold as well as the business. The wisest businessman will only own the land, period.
Well, it is certainly fun to rummage through the server dust bin and cobble together a storage cluster based on open source software, but who would seriously consider using it as a production storage tier in their organization?
You can get a very low TCA if you build your own white box storage servers or turn to ODMs like QCT and Supermidcro. Google started out building all of its servers as cheaply as possible by doing it themselves. There is really no reason to start with junk servers unless you need to prove the concept before you get the funding you need to do it right.
DIY open source storage software like Ceph is not a walk in the park if you don't have a computer science department nearby. Ceph has a complex, non-P2P architecture with no built-in capability to do charge-back, QoS or reporting...stuff that people are interested in having. Upgrading Ceph has also been difficult. Maybe Red Hat has made some progress along these lines with the release of Red Hat Ceph Storage 2 this past June.
I do agree the determining the TCA for a capacity storage project is relatively easy and coming up with a fully burdened TCO is more difficult, but not impossible. Personally, a single storage administrator should be able to manage 10PB of objecgt-based storage, assuming the cluster is not made of junk servers needing constant attention.
"There is really no reason to start with junk servers unless you need to prove the concept before you get the funding you need to do it right."
As noted in the article. These things usually get thrown up for third-string stuff that was just handy to have and tend to grow organically into the organization.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
