Re: open source version of XIV
Do you get the same sort of performance from Google Cloud though?
The grid structure is aimed at two problems which affect RAID-based arrays: performance and rebuild times. With, say a 7+P RAID 5 array only 8 disks are able to serve I/O. Overall, those traditional systems will balance their I/O across all of the disks as they should all be in play if the system is loaded as a whole, but for a particular workload you're limited to one array; 8 disks. With XIV you are guaranteed that everything is spread everywhere.
Rebuild times are arguably more of an issue. If you have large disks (people are only buying large disks + flash nowadays, i.e. the XIV model - 15K is dead) then the time taken to rebuild a RAID 5 from 7 disks to a hot spare is horrendous, especially if the array is still serving a lot of I/O. RAID 6 offers an extra layer of protection, but like RAID 5 there's a write penalty, only it's worse as an extra read/write is required every time you write new data.
Distributed RAID has helped to overcome the performance issue, but be aware that some implementations don't distribute everywhere like XIV does, It also still suffers from the write penalty, just like it always did.
Of course, the downside with XIV is it's hopelessly inefficient in its use of capacity: a compute node for every twelve drives and basic mirroring. Add in spare capacity and you're already below 50% efficient. Even with cheap drives you're using a lot of space for not a lot of capacity.
Moshe's latest, Infinidat does take it a step further, using a form of distributed RAID which is more like the erasure coding you get in cloud storage. It's fixed at two parity blocks, so like RAID 6 but spread over all the disks in the system like XIV. They get over the penalty by writing a new block rather than updating the existing one.
As for open-source, object storage implementations like Ceph will do erasure coding but they're not really suitable for enterprise block (Ceph will do block but you'll need to throw flash at it to get it to perform, plus you need an army to set it up and maintain it, relatively speaking). Can't think of any block SDS implementations.
Biting the hand that feeds IT
