SSD Caching perhaps?
It would be very tempting to include some Flash Cache on any HDD.
If these limits are new, then perhaps something like that is the reason.
A recent Register article reminded me that hard drive vendors are introducing warranty limits on their products for the amount of data that can be written to them in any one year. SSDs have always had this restriction; does the same now apply to HDDs? Solid state drives (SSDs) have always had this restriction because we know …
The Seagate consumer drives with "Flash" cache sucked, to put it mildly. I know because I had three of them, two in a RAID enclosure. They all failed within a week of one another, just before the end of the warranty period. The replacement drives were DOA or died within weeks of arrival. So three replacement drives for nothing. Thanks for the "replacement with fully tested, refurbished products," Seagate! I wasn't going back through the hassle of another replacement cycle just to get more defective, useless product.
The original drives had become severely sluggish, probably due to the RAM cache wearing out from excessive writes. I never saw the advertised significant decrease in boot times, faster application launches, etc. In concept this seems a good idea, but I wouldn't ever spend the money with Seagate again, not even for a regular HDD. Their product was crap and the "warranty replacement" process and products were complete failures. If you need more confirmation, look at the reviews on amazon.com (US). The product was trash.
I'm saving up to replace all my HDDs with SSDs; no more "hybrid" drives, ever. It frightens me that Seagate's hybrid drives were ever targeted for enterprise customers. Zip drives were more reliable back in the day, and that's not saying much, obviously.
but how much will it cost?
That seems to be the killer.
Yes the price has been coming down but go above 1TB and you have to pay an awful lot for the device.
I wonder if there is a market for large capacity flash but with not 'shit off a shovel' performance. I'd go for it.
slower and cheaper per Gigabyte could make very large drives (should we continue to say drives?) affordable.
For HDD, you save a bunch of money if you accept slower I/O, because a slower motor is cheaper and allows cheaper I/O electronics, and a slower actuator is cheaper. Furthermore, increased parallelism is very expensive. Access electronics are very expensive because of the complicated analog circuitry needed for the magnetic signals.
By contrast, you save almost no money if you accept slower I/O for flash. Access electronics are cheap and therefore any amount of parallelism is cheap. Throughput is limited only by the speed of interface. Read latency does have a lower bound but is thousands of times better than HDD even for very slow flash. Flash cost is driven (to a first approximation) by the number of flash bits. Large-scale flash storage cost will closely track the cost of flash chips.
"Furthermore, increased parallelism is very expensive."
There's no parallelism in a HDD. Only one head is ever active at a time.
Parallelism was tried and abandoned as "too hard" a long time ago. It's possible that WD or Seagate might revive the technology but as it wouldn't improve IOPs (only sequential throughout) there's not much point.
There is parallelism, but not in a single device. Striping allows for higher throughput on sequential access. It also allows for more random I/Os in proportion to the number of devices, but in aggregate only. Many people have been there - thrown more disks at a workload in an attempt to get more throughput, often at the expense of leaving a lot of unused space (using short-throw mapping techniques to put all the data on the outer tracks of a disk which minimises seek latency and maximises sequential throughput).
All this costs money and has a big power cost too, but it is parallelism. You might conceivably solve aggrefate throughput problems with parallelism, but it can never fix the latency issue.
No, there's a limit there, too, as more platters will strain the spindle and the motor. Old drives in the past spun slower, reducing the forces but also lowering the performance. That's why Quantum's brief step back to 5.25" hard drives fell flat. Eventually, as noted in the article, rust is going to run in to the immovable wall of physics AND be pinched by performance demands (I can speak from experience. Mirroring 3TB worth of stuff over USB3 took the better part of a day; transferring lots of data takes an unavoidable amount of time which opens the door for reconstruction failures) that prevent larger but slower solutions.
Found the article that is the source of that image.
That's an IBM 3390. $250,000 that thing cost. In 1989 dollars. Yup, the thing is a quarter century old and held somewhere up to around 22GB, which doesn't seem much until you realize at the time, 200MB hard drives were just coming on the PC market and were no small change, either. So it kind of solidifies my point, as it's very old and very expensive.
Taller drives = higher spec motors and more platter flutter. This won't happen and larger platters won't happen for similar reasons.
Fly heights are 1/10 what they were when drives broke the 100Gb barrier. There's simply not the engineering tolerances available to allow taller drives/larger platters, even if drives were slowed to 3700 or 4200rpm again.
The joke is on you: these drives were designed for Windows '95: enough capacity for Joe's porn videos, just fast enough to play video and the reliability of a politician. Enterprise disc drives don't come with rate limits, are fast and the failure rate is ten times lower.
Come to think of it: if discs are made with SSD-like lifetime limitations, that must mean the manufacturer put an SSD into the box as cache.
> However, there’s no direct information in the spec sheets to say drives are warrantied for data written. In fact, terms such as “designed for” are used more often, so where do we stand with the warranty?
In Australia, it's actually pretty simple.
Companies can include or exclude whatever they want; it doesn't reduce makes no your rights under consumer law. Unless that writes/year is clearly stipulated in the box, visible before you make the purchase, they can't enforce it (won't stop them trying of course). They don't even provide an easy way to measure how much has been written, so it would be difficult to say the least for them to enforce even if they suspected you were "naughty".
Whilst it's all very nice being able to squish TBs of data into the usual 3.5/2.5" packages I'm always a bit squeemish about getting the latest/greatest/biggest hard drives. Though on whatever article it was the other day, I wasn't expecting write limts (or whatever) on a hard disk (opposed to an SSD).
Not really used SSD myself in a big way. At work we've got a few as 'expendable' scratch area media. At home my iMac has one of Apple's Fusion drives in it, which seems to be fast enough. I'm not sure about laptops with only an SSD in, I giess as long as they're easily user-replaceable then there's no reason why not.
>Really not sure why you think SSD's in laptops are a bad idea?
Probably due to yet another employer being penny wise but pound foolish. Take the taste test. Buy yourself a fairly cheap $70 ssd drive (can get 240gig for that online) and use it as a boot drive for a week on any Winows 7+ machine (or Mac or even shudder systemd Linux) and you will never want to boot off spinning rust again. It makes hibernation unnecessary.
Because when SSDs go tits up they sink to the bottom of the pool HDD's tend to float on the surface. SSD are wonderful most of the time but if you have an "event" then the chances of getting up and running without a complete re-image are not good.
On the plus side - it one takes one "event" to persuade management that backups are a really good idea so there's a silver lining to this - SSD's mean much faster performance and eventually lead to better backups (on HDDs).
I've installed 6 SSDs (the oldest is a 256GB Crucial bought almost 5 years ago - and very expensive it was). All are still in use (and some in their second machine). In contrast, I've had perhaps a dozen HDDs and I've had three sudden failures. Note that not all these failures were complete, but the disks became essentially unusable due to unrecoverable errors. Maybe a specialist could get the data back with the right equipment, but I couldn't.
My experience (admittedly not a statistically large selection) is that SSDs have been very reliable and that HDDs can, and so fail suddenly. In any event, the first rule of IT is make sure you can recover everything important. Don't ever rely on a single device.
MacBook Pro, 15", 2015 model with SSD: blows the previous model I had out of the water, speed-wise. The SSD is warp speed, all the time. I don't want a spinning rust drive in my primary laptop again, ever. The only problem was that the maximum size offered by Apple was 500GB, which barely contains my music and photo libraries. Now that I've learned the SSD is upgradeable by the consumer, I'm upgrading to a 1TB SSD, the capacity I had in my previous MBP.
"so where do we stand with the warranty?
I need to check some of the detailed product sheets. "
I have the same question, and I agree you should have checked and included the information!
Regarding rebuild times, this is essentially irrelevant. The problem with large drives is with recovery of the data not with time taken to do so. With the expected unrecoverable error rates of current drives you probably couldn't even read a whole 16TB disk successfully in one go, so mirroring is useless, RAID 5 is useless, and RAID 6 is probably useless. Erasure coding might help.
Reliability could be improved through internal data resilience at the cost of some drive space. Many "drive failures" are not actually drive failures at all so internal resilience could be used to recover some of the UREs.
"RAID 5 is useless, and RAID 6 is probably useless. Erasure coding might help."
Raid6 is almost usless - we've lost raid6 arrays during rebuild.
That's one of the reasons I moved to RAIDz3 a few years ago. the other being that ZFS is very good at detecting errors (far better than the 10E-14 that HDDs have for ECC) and flagging drives which are playing up long before it shows in SMART stats.
... the read-writes that go on under the covers performed by most RAID controllers to prevent bitrot. It could very well be that there is further 'amplification' (and, it should be noted, will also happen as long as the RAIDset is powered, even if it is not being actively written to.
This probably makes it even more important to not buy all of the disks in a RAIDset at the same time or from the same batch of disks.
The argument for selecting drives from the same batch for a RAID is another big question.
a) The "probability" of having a dodgy production disk increases when you have disks from different batches.
b) The "probability" of multiple concurrent disk failures increases with disks from the same batch.
More single drive failures or risk of a total data loss? :-) I've always gone for a single batch, but with that RAID backed up to a totally different device (hardware, software and drives).
Ah, but more frequent single failures in a raid set is an annoyance, but not putting your data at risk (as long as you replace the failed disks)
Multiple concurrent failures risk your data!
I will opt every time for a scenario where I have to replace single drives more frequently, as opposed to one with less frequent work, but increased risk of data loss.
The limits seem to be arbitrary absolute data values and not some product of the capacity of any particular model... so it seems to me that the spinning rust pushers are trying to indemnify their "warranties" against covering normal wear to mechanical head/arm components and the like, or some newfangled flash buffer as JP suggests, rather than the plates of rust themselves.
Strikes me as pretty slimy.
Warranties never cover normal wear and tear, they just cover manufacturing defects. If they're now adding expected read/write lifespans to their drives what you should be asking is how these values compare to previous models. Then you should buy the drive whose warranty covers the lifespan you need ;)
Mechanics. Large platters wobble more so you can't have the tight tolerances required to get data density. The tradeoff is worth it, hence we've not had physically big disks for a long time. They could make them taller, but there would be no point as you can double capacity with two drives that way anyway.
Clever people have increased density by adding drives through to the back of the chassis, and larger drives wouldn't help there much if at all.
If you think it through, and read my comment above, we've actually hit a point where larger drives are not desirable. I'd rather see smaller form factors with greater density but lower capacities. This would allow sensible data protection schemes and less disruptive drive replacement which are very nice in the petabyte scale!
... why do drives have to be limited to 3.5 or 2.5 inch sizes? Using the larger 5 1/4 inch size would allow far more data to be stored ...
The larger the platter the more energy it takes to spin it up (and down) and the greater the gyroscopic force on the spindle (causing wear) if the disk is moved while spinning. Large platters also need longer, and so stronger, and so heavier, arms for the read/write heads, so these have more inertia, which increases the energy needed to move them and increases the track-to-track access and head-settling times.
Also, the more energy you need, the more cooling you need ...
There's a reason drive sizes have been getting progressively smaller and smaller.
I recall 14" drives which span at 3,600 RPM. Now just try imagining what the latency and seek time figures looked like. Now try engineering such a beast with current data densities and imagine how many 10s of TB there would be to access with such slow access speeds.
Don't think you can spin this thing any faster. Try it and you'll find the forces are such that the aluminium platter will stretch at the outside and the platter will ripple. That's let alone trying to speed it up to the 15K RPM you see on the enterprise drives. By that time the peripheral speed is approaching the speed of sound (I suppose helium enclosure might help).
nb 14" drives running at even 3600 RPM could be dangerous - I'd heard of one in a data centre where the bearing sized, the shaft sheered and it wrecked several cabinets. There's a lot of kinetic energy in a 14" platter spinning at 3,600 RPM.
There's a reason why drives got smaller and smaller...
"That's let alone trying to speed it up to the 15K RPM you see on the enterprise drives."
It's worth noting that virtually all 15krpm or faster drives use 2.5" or smaller platters - made of glass.
WRT kinetic energy: Not HDDs but a couple of other examples of spinning things running amok:
A centrifuge in a university biology lab had bearing failure at 10,000rpm. The 10kg rotor exited the building via a concrete block wall (5th floor) after blazing a trail of destruction across the lab. Thankfully it landed in a lawn and dug itself a large hole before it could travel any further.
In the 1970s a 2MW hydro turbine lost lubrication and exited the generator room after levering itself out of the concrete floor. It was found several miles downstream. Had it gone in the other direction there's a good chance it would have destroyed the dam.
We have a stack of 6 3TB Seagate surveillance HDDs spare, warranty replacements from 6 that failed in quick succession in a Ceph cluster.
The actual drives in the hosts were replaced by consumer Hitachis prior to the duds being sent back to Seagate. We haven't been game to try the replacement disks in any machines yet. So far, the Hitachis have out-lived them.
You never want to use drives from the same batch in a RAID set, because of the massive difference in per lot failures. With different lot drives versus identical drives, you are less likely to have second failure after the first. You can't guarantee different lots, but you could increase your chances by buying from different places, instead of ordering them all from one spot.
I consulted for a company once that had a lot of small RAID arrays (JBODs attached to servers running software RAID) that generally ordered drives in batches to build multiple servers. They'd sort through them to match up dissimilar lot numbers (and ideally factory locations) and send them back unopened for replacement from their VAR if they had too many from the same lot for their mix-n-match strategy. I was a bit skeptical of all this at first, but they had a rather anal guy who developed this policy who kept track of drive failures in this way since he started working there, and sent me a big spreadsheet to prove his point :)
Biting the hand that feeds IT © 1998–2019