Mr Pott
I find myself quite envious of your home lab
Supermicro has released the first of its new line of Twin series unblade* servers and El Reg has taken the opportunity to given them a right good kicking. The 2028TP-DC0FR and 2028TP-DC1R models have crossed our lab and here's what's what with the 2015 Twins. As would be expected when designing a line of mostly similar …
You only need the SLOG if you're pushing sync writes - and you're better off using something like a dedicated 16Gb HGST S800z for most purposes or a 8Gb HGST ZeusRAM if you _really_ need blistering iops (Remember: the SLOG is _ONLY_ there for power failure recovery. Under any other circumstances it's a write-only circular buffer and even in the event that the drive goes titsup at the same time as the power goes off, you only lose the pending writes, not the entire FS.)
(It'd be interesting to see a small fast flashdimm in these applications.)
As far as dedupe goes: DON'T DO IT. At least not for more than a couple of hundred GB of storage
ZFS requires about 1GB ram per Tb of disk under most circumstances. Adding dedupe will bump that to 4-6Gb/Tb with the requirement scaling exponentially as storage increases. If you skimp on ram then ZFS works well until it doesn't, at which point it really _really_ doesn't.
Right. I am well aware of RAM requirements of ZFS deduplication which is why I'm not using it and not recommending it. With this out of the way, lets talk about SLOG.
As mandated by POSIX, ZFS will by default complete all synchronous (as requested by caller) writes before returning to caller. Also, any metadata changes in ZFS are performed synchronously. Many filesystems use journal for this purpose, ZFS uses ZIL, i.e. ZFS Intent Log which is either carved from storage space of your volume, or placed on dedicated volume depending on 1) presence of "log" device 2) option logbias . Also, ZIL can be explicitly disabled (thus making filesystem behaviour for synchronous writes non-compliant with POSIX)
Now, assuming that ZFS setup has not been "optimized" either by "logbias=throughput" or "sync=disabled", there is big benefit from having dedicated log device with low latency, because that allows all synchronous writes to complete after intent has been written to such dedicated device (as opposed to writing to data volume with large latency). Looking at latency figures, ZeusRAM is up to 0.023ms and Intel P3700 is around 0.02ms (however, ZeusRAM capacity is only 8GB and P3700 starts at 400GB - which leaves lots of space for other purposes such as L2ARC, however we do not know max latency of P3700 only average). This number should be compared against latency of spinning rust storage (or whatever is used for main data volume) which typically would be somewhere between 2ms - 12ms depending on specific HDD in use. This means that synchronous writes would complete much, much faster if dedicated log device such as ZeusRAM or Intel P3700 was used. Normally this could significantly boost IOPS number.
However, what we do not know is whether 1) Maxta does actually use synchronous writes 2) its underlying ZFS ZOLVs are not "optimized" to avoid using ZIL. It would be interesting to learn this.
A negative is it's supermicro.
They have their use cases, but not in my datacenters.
I'll take iLO4 over ipmi in less than 1 heartbeat. My personal supermicro server's kvm management card is still down since last FW upgrade a year ago. I have to go on site and reconfigure the IP. Fortunately i haven't had an urgent need to.
Looking forward to my new DL380Gen9 systems with 18 core cpus and 10GbaseT.
"By now, there's more than a little of my blood inside those nodes."
Something like this might help:
http://www.magidglove.com/Ansell-SafeKnit-72025-Ultralight-LightDuty-Seamless-Gloves-1-720256.aspx
They're made of spectra, so shouldn't be too thick/bulky to be able to feel anything.
I like the gloves as well (and I was going to quote the same line ;-). On the other hand (I'm an economist, so it's allowed) the work inside my systems where I do not lose some skin and blood never come out right. I have to keep going until I've made my sacrificial offering to the Quantum Gods. Or Whomever.
At least They don't require a burnt offering. Did that with -20kV. Levitated. High. Very, very high. But the VORTAC worked fantastic for the next three years after getting repaired. [Engineering did a reverse power test where the breaker did not break.Oops.]
On rails, Dell are indeed the best I've seen. HPs are easy enough to install but removing them is a bloody nightmare - especially for racks of 1U servers all with cable management arms install - where there is very little room to get at those bastard tabs on the inside. Whoever thought that was a good idea should be made to rack and unrack servers until they understand their mistake.
I remember the last time I moved to a new data-centre; my fingers were actually bleeding and sore for quite a while afterwards, including a blood-blister under a nail.
I like my server stuff made out of metal but those little blue plastic tabs on the Dells get high marks from my fingers. A recent replacement of an older HP with a newer Dell (that was a prep rack) was informative - one person and half a minute to remove the rails for the Dell; 2 people, a screwdriver (for leverage and jamming into your hand) and 5 minutes for the HP.
Thanks Trevor - insightful and entertaining as ever.
Can anyone vouch for Supermicro in the real world? We're a MSP and usually resell Dell or HP and bang on Windows, but I'm giving serious thought to a 1U Supermicro + enclosure with Windows Storage Server 2012 X 2 as a single solution starter SAN... Looking at < £10k and it would be the same storage plus additional features as a HP LeftHand starter SAN we resell for over £30k!
I'm comfortable with Win Storage Server doing its bit - bit we've never used Supermicro before and wondering what we're missing...
Thanks - Steve
The SuperMicro kit I've used has always been very reliable, and pretty well made.
Be aware that in general 1U systems are _very_ loud though (all brands). If noise is an issue, a 2U system with the same general specs will be much quieter. All generalising, but keep it in mind if needed. :)
I have over 5000 Supermicro servers deployed across my client base, in addition to what's in my lab. The failure rates aren't particularly high, and I did have a very nice conversation with the Supermicro CEO yesterday that completely put to bed al my questions and concerns about their enterprise support. So, I don't know how much more real world you need...ask a specific question, I'll give you a specific answer....
I second the motion on how awful working inside Supermicro hardware is. I have a couple SC847 storage server cases in my lab, and getting into them to recable stuff is a nightmare. even removing the motherboard subchassis, UGH. By contrast, I had to swap a mobo on a HP proliant gen8 box and it was a breeze, not a sharp edge in site.
beer, because I need a couple strong ones after wrestling with the SC847
Those aren't "storage server" cases. They're crap which transmits too much vibration between drives to be usable under high seek loads (and they're far too big for the number of drives they hold)
There's a lot to be said for Dell's 60 drive JBODs (which are rebadged EMCs as far as I can tell) or if you're skint, Infortrend's JB2060s