back to article Dell soups up low-end Data Domain deduper

Down in the Dell there is a new Data Domain box – a smallish, entry-level one – the DD3300. It is a 2U rackmount chassis with 3TB or 4TB SAS disk drives providing up to 32TB of usable capacity, said to be 1.6PB after deduplication (logical capacity) and data can be ingested at 7TB/hour using Data Domain Boost. If a backend …

  1. W. Anderson

    What software runs this Dell storage?

    I would be curious to know what software technology Dell used for this Storage technology, since it is generally understood that ZFS is the indisputable OS File System, and large capacity storage software like Ceph are the standard, while Dell is predominantly a Windows software only vendor.

    1. Anonymous Coward
      Anonymous Coward

      Re: What software runs this Dell storage?

      ‘Indisputable’, only in circles not familiar with the alternatives. Isilon’s OneFS, Nimble’s CASL and Nutanix are just three of the file systems that would make ZFS a very disputed file system OS ... DD doesn’t use any of these. From the little I’ve seen DD looks Linux based, might be ext4 but I don’t think it’s ZFS, which has some challenges, poor snapshots, performance dropping when over 70% full, complex to manage, certainly disputed (not that I am saying Ext4 is perfect). It all depends on what you want your file system to do, for many uses ZFS may be adequate, but it’s not Elvis.

      1. Nate Amsden Silver badge

        Re: What software runs this Dell storage?

        I migrated my org off of ZFS(Nexenta on OpenSolaris) for backup/dedupe onto HP StoreOnce (I believe filesystem is IBRIX but dedupe was in house HP labs stuff). Averaged 6:1 dedupe on ZFS(July 2014 from my records here), shot to about 37:1 on StoreOnce (both systems accessed over NFS). Current dedupe rates across a few shares:

        share 1 20TB user data 39:1

        share 2 2TB user data 22.6:1

        share 3 781GB user data 12:3.1

        share 4 500Gb of data 33:1 (not much active data here)

        share 5 8.3TB user data 62.4:1 (I think inactive data here too)

        share 6 13.5TB user data 13.6:1

        This is on the smallest HP storeonce appliances (1U box / 5.5TB raw). HP's marketing stuff references 20:1 ratios a lot so I think we get pretty good ratios all things considered, the last share has a ton of unrelated things stored on it so dedupe is lower, there is a limited number of shares allowed, and the dedupe applies at the share(file system?)

        typical data retention above is 60 days. StoreOnce is far from perfect, NFS performance is not good, maybe get 5-10% of the throughput of the super accelerated VTL stuff(for bigger backups have to mount NFS with sync mode which slows it even more). But it works for our needs.

        ZFS dedupe on linux at home I tried recently(debian 8), completely unusable, on a 4-disk raid 10 on 3ware throughput was under 1MB/sec(32GB of ram quad core intel Xeon). No idea why. I created a 100GB volume on the 3ware(LUN level not LVM or anything) because I had no need to run dedupe on the entire array(especially knowing ZFS's memory needs for dedupe). I was hoping to replace rsnapshot with zfs dedupe, but after about an hour I gave up and went back to rsnapshot (still on ZFS but just no dedupe). Without dedupe performance easily 30-50MB/sec(don't care about going much faster than that).

        I have no doubt zfs dedupe on linux can work fine, not sure what the issue on my side was, wasn't important enough to try to fix.

        Not sure if DD's NFS is much better or not, seems everything is optimized for VTL.

        1. Anonymous Coward
          Anonymous Coward

          Re: What software runs this Dell storage?

          Any Windows 2016 box with ReFS will give you more for a fraction of the cost. Bulk storage servers are available through HP (Apollo series) or Cisco (S-series). The latter is leading in throughput.

    2. M. B.

      Re: What software runs this Dell storage?

      Data Domain OS (aka DD OS) is Linux-based, runs on top of proprietary SISL file system to get the inline deduplication performance required for ingesting that much data onto so few spindles.

      Restore times can be a bit LOL though...

    3. SnapperHead

      Re: What software runs this Dell storage?

      Learn grammar to get your point across clearly.

  2. kryptylomese

    fdupes anyone?

    Linux "fdupes" does this

  3. SnapperHead

    The most expensive SATA drives money can buy. And they grow like mushrooms.

    1. Michael Duke

      XIV wants you to hold its beer. :)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019