Array reliability factors
The trouble with single parity is when you are recovering, you have no parity. And it's at that time you discover thata block on one of the other disks has gone bad and can't be read. Ouch.
Data-scrubbing helps. (That means that the array reads itself in toto every night or every week, while there is still a working parity disk. If a read error happens, don't fail the disk immediately. Recalculate the block from the other disks and re-write it. For a single bad block, the drive will re-map it and that's the temporary end of the problem, with the block now stored elsewhere in the drive). You are of course also watching the SMART statistics, and if any Reallocated sector count starts increasing other than very infrequently, you pre-emptively replace that drive )
Even so, you'd prefer to have parity during a rebuild operation.
If it takes a day to rebuild and the expected time to failure of another drive in the array is 100 days, that's a 1/100 chance of losing the multi-Terabyte array with RAID-5 after the first drive fails. That 100 days is not 3+ years, because the first failure may reflect a common defect in the whole batch of disks. It may well be less than 100 days to the next, fatal, failure.
With Raid-6, two more drives have to die for it to be fatal, that's 1 in 10,000 chance . Probably good enough. Three parity disks gives you 1 in a million, even for a fairly pessimistic MTTF assumption. I think SUN has this about right.
I wonder if any RAID array supplier has ever bought disks in batches well ahead of time and kept them "hot", so that they could ship arrays to customers with no two disks from the same batch, and all with different run-times (spread over several months) at time of assembly? Supplying somewhat "used" disks would actually considerably increase reliability!
I used to build 4-5 disk Linux software RAID arrays using one disk made by each of 4-5 manufacturers to eliminate common-mode failure as far as possible. But today, there are only 4 manufacturers: Hitachi, WD, Seagate, Samsung.
As for always using Raid-1 or -10, it isn't much better UNLESS you always mirror to a drive made by a different manufacturer. If both drives are from the same batch, common mode failure, failure of first drive may imply MTTF of ther drive is 100 days. 1 day to re-mirror -> 1% chance of total loss. Ouch, again.
If you (or your vendor) has had the foresight to mirror to a different make of drive, failures should not correlate and MTTF of the surviving disk is probably 1000 days, 0.1% chance of total loss. RAID-6 is more reliable. Of course, performance isn't as good. And these are post-first-failure probabilities, some folks will never see any drive fail at all.
BTW how many folks out there are using hardware RAID controllers with non-partity RAM buffering your data? So a faulty memory chip goes undetected until all your data is scrambled?
Another win for Linux software RAID, just as long as it's running in a server with ECC RAM. Non-ECC RAM in controllers is IMO the reason why RAID-[5,6] has a bad rep.
Maybe it all becomes irrelevant soon. With the latest filesystems you will be able choose per-file or per-folder-tree what level of redundancy you want. Multiple partity with backups for the filesystem metadata. RAID-1, or -6, or -5, or none for files, depending on how important their contents are deemed to be. Also end-to-end checksumming to detect faulty controller electronics or buffer RAM. Also self-healing. So systems won't want RAID controllers at all, they'll just want a big pool of raw disks for the filesystem to manage.
Sun started this ball rolling with ZFS, a shame they seem to have dropped it of late.