back to article Panasas: Avoid lengthy RAID re-builds - use our dodgy-file tart-up tech

Storage biz Panasas has unveiled boosted hardware and software – and attacked the RAID rebuild problem by rebuilding damaged files instead of complete disks. The company is an HPC storage system supplier which is moving into technical computing/big data use cases in enterprises. It has more than 450 customers in 50+ countries …

COMMENTS

This topic is closed for new posts.
  1. pierce

    ZFS works much the same way.

    1. Anonymous Coward
      Anonymous Coward

      Are you a pr troll?

      You want to sound important, but you sound ignorant (like the BB trolls I have encountered). Ceph does erasure coding. ZFS does RAID.

  2. Bronek Kozicki Silver badge

    I can see a weakness here

    Yes you lose less data the more HDD you have in an array, but also the probability 3rd disk failing in your 100 disk large RAID 6+ setup while your files are rebuilt is also larger. This is even if you only rebuild files damaged by preceding failure of 1st and 2nd, because the probability of 3rd failing is proportional not only to rebuild time (nice that they made it shorter, though) but also to the number of drivers you are left with.

    1. JLH

      Re: I can see a weakness here

      Panasas isn't a RAID6 array. The individual files are stored as RAID objects.

      So if you lose that hypothetical 3rd drive you don't lose everything.

  3. Suricou Raven

    HE6 drives?

    Have you seen the price of those things?

    For some enterprise users, money seems to be no object.

  4. Joerg

    A new RAID level that scales redundancy...

    RAID-6 has around 30% redundancy with 8 drives .. which means that 2 can fail ...

    ....the problem is that if you got 16 drives that doesn't scale up to 4 drives that can fail.. you are still limited to just 2 .. BUT redundancy is still around 30% and on 16 drives that should mean 4 drives not 2 ...

    THAT is the biggest design flaw of RAID-5 and RAID-6 algorithms...despite all the redundancy being used the number of drives that can fail doesn't scale up !

    So you need to add hot spare drives (full redundancy) to your RAID arrays just to lower the chance of the whole array failing during rebuild.

    Why the industry hasn't come up all these years with a RAID-7 standard addressing this huge design flaw it's a real mistery. The algorithms are really complex but the redundancy it's there, the design just needs to be fixed to make full use of redundancy properly scaling up with an increasing number of drives.

    1. Bronek Kozicki Silver badge

      Re: A new RAID level that scales redundancy...

      I like what you are saying, but there is one RAID schema where redundancy keeps up with the number of drives added. Unfortunately it's also the most expensive of them all, it's called RAID10. I'd preferred if they developed what you call "RAID 7".

      However, the more interesting thing here is that by coupling HDD redundancy logic with the filesystem they repeated what ZFS does and to a good effect, i.e. much shorter rebuild times. The rebuild time is when your data is most vulnerable, so making it shorter also helps.

      Also, if this trend is followed then perhaps live snapshots, checksums and self-healing will be too. I hope I live long enough to see these in general purpose consumer hardware - delivered and enabled by default.

    2. JLH

      Re: A new RAID level that scales redundancy...

      Joerg, I have installed and run Panasas systems.

      It is not a RAID 6 storage array.

      "What Panasas’ RAID 6+ does is to end the default rebuilding of an entire failed drive and only rebuild the damaged data components.It can do this because, at heart, it is an object-based parallel filesystem. "

      ie you rebuild the objects which are on the blade which has failed (or indeed you can move data off a blade which is suspected to be nearing failure).

      Small files are stored RAID 10, and larger files at RAID 5 or RAID 6 with this new array.

  5. JLH

    Its worth explaining what Panasas is. It is a 'storage cluster'.

    Each storage blade int he rack is an independent computer - with a processor, RAM, flash storage and/or spinning disk storage.

    Small files are kept in two copies, each on a separate blade.

    Larger files are stored across a set of blades.

    So if you lose a blade, the blade is replaced and a new blade is automatically installed.

    1. Bronek Kozicki Silver badge

      Thanks for explanation - I can see that RAID6+ is really misnomer here ..

  6. Anonymous Coward
    Anonymous Coward

    Panasas really missed the point!

    At $160,000, Panasas is using a monster truck to deliver pizza. If the object is to make files available quickly, this is not the way to go.

    For the money, you could build a 1 PB object store with 3-way replication and have plenty of change left over. At $39 for a 1 TB drive, replication is really cheap. The hard drives for a 122TB PAS equivalent would cost $12,000 (3 replicas) and the 10 or so server/appliance boxes would add less than $30K. Throw in Ceph and things look way cheaper.

    The trend is away from big complex hardware solutions!

    1. Anonymous Coward
      Anonymous Coward

      Re: Panasas really missed the point!

      How long have you been in the IT industry? That's the LIST PRICE. Do you think Panasas would be winning multi-petabyte deals in highly competitive situations at list price level? I suspect that you are a technical guy, and so you should leave the commercial aspects of the job to your management.

  7. disk iops

    article's "Raid6+" is probably wrong

    a recent academic paper (FAST '13) cited a Fast '08 paper by Welch, Abbasi, Gibson et. al. explained that Panasas used 12+4+2 erasure coding or was it 14+2+2. (pyramid codes). Likewise ;login of 12/2013 by James Plank (highly recommend reading it). In any event, nobody who wants to stay sane uses RAID. It's all erasure coding all the time. Each object is independent coded when saved. Each hard disk is just that, a simple hard disk.

  8. Hank Hill

    Why such a confusing title?

    Mr Mellor, I must call upon you to explain the title of your article. Are you saying, as I suspect is the case, that the Panasas technology can take broken and corrupt ("dodgy") files and repair them ("tart them up")? Or are you saying that the Panasas File System is dodgy and that they are a (s)tart-up company? I'm sure the latter is not the case, as Panasas has been shipping this technology for over 10 years now, with a large and impressive customer portfolio, and so this is most certainly not "dodgy" tech from a start-up company.

    With respect, I'm sure you would like to think that your articles are read by an international audience, so I'm quite baffled why, as a professional journalist, you would use such Arthur Daley-style vernacular in the title, which would only serve to confuse anyone outside the UK.

    1. gazthejourno (Written by Reg staff)

      Re: Why such a confusing title?

      You've not read much of the Register before, have you?

      Signed

      A headline writer

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2019