Give me 10 gig Ethernet now!

ATM

I guess if there was any life in ATM, it's surely extinguished now...

Good luck with that

After moving into our very shiny new offices, refitted with "all the latest kit", I soon found out after a trip into the server room that our company networking guys had been cutting corners to save a few bob. Not only were we under required capacity by about 50 fecking ports, but one switch was gigabit, and the other, identical-looking switch was actually 10/100Mbps. Cue phone calls and swearing.

If they're going to try and save such a pitiful amount (these were not large switches) because *gigabit* is apparently too bloody expensive, 10G has no chance.

The key is to reduce the amount of data being backed up in the first place rather than simply throwing more bandwidth at it.

Err...

No mention of disk snapshotting, so you can backup offline?

At my previous company we had about an 800TB overnight backup, in order to achieve this we made heavy use of fibrechannel attached tape drives and disk snapshots mounted up on dedicated mount servers that could backup offline. Just before I left we started to use 10GigE IP clients on mount servers and left the tape drives at the general purpose storage nodes/media servers. This way the storage nodes/media servers could be shared with normal 1Gig backup LAN clients and 10GigE mount servers. It certainly drastically reduced costs in licensing the mount servers.

Manage your data properly

main issue we have is with users not managing their data properly. We've got around 750TB to backup. When you have users moving data to areas that get backed up when that data doesn't need backing up, or just moving stuff around that then triggers a full back up its a problem, well its a problem when these datasets are 15TB or more!

Is 10GbE much faster than 1GbE?

100MbE is ~10x faster than 10MbE, but actual transfer rate is rarely 10x between 100MbE and 1GbE. Most of the time I see 4x, due to the TCP/IP overhead, even using Jumbo Frames.

How much of an increase in transfer rate do we see between 1GbE and 10GbE using TCP/IP?

Is it time for a new protocol?

overhead and 802.3ad

10gig is cheap, seems crazy to me that just a few short years ago a line rate 48-port 10gig switch would of cost you half a million $ and a good chunk of a rack and now it can be had for probably 93% less and take up 1U.

As for 802.3ad which the article mentions as a stop gap - in many situations it is not because the algorithms that load aggregation uses often ensures that even with multiple links only one link is used if the data transfer is between two systems, works better if its many:many or at least many:one.

Another issue is stress on the storage, my last big storage array our budget was limited so we ran it pretty hot. We had enough to backup what we needed (about 5-6TB/week if I recall right), the disks didn't have enough spare I/O capacity if we wanted to pull many TBs off per day (the array itself was 100+TB usable).. We ingested data at a rate of about 5TB/day, fortunately our backup needs were much less than that.

10Gb too expensive, need better backup

The first problem is with the setup. Instead of having just one or two ports to the great outside, put the tape on the "inside" of the storage server. Remember, that's what we used to do?? There are plenty of non-networked options for that. A SCSI Ultrium 5 drive does 140MBps native, so a full tape takes 2 hours. Stripe them, and that multi-terrabyte array will be backed up awfully quick.

The other part is good backup software. When a company doesn't care about its backups, the you're lucky if you even use the provided backup software. The other part is using software which works with how you work, for a minimum system load.

Sure, we have more protocols, but the problem is the software, not the stack on the OS. How many times have you seen something supporting SCTP? At all? That's where things go awry. (I've met more than one programmer who had never heard of TCP "out of band." Ouch.)

36U?

Where's your datacentre? We haven't had the power budget to stuff a rack that full since the early noughties!

Besides, we abandoned top-of-rack switching years ago so we don't suffer from that bottleneck anyway.

Details, details

"Consider for a moment a fairly typical backup requirement of 10 terabytes. At gigabit Ethernet speeds, allowing for 10 per cent network overhead and assuming zero media changes, it would take a little over a day to back that up."

OF course you could use some modern technology to improve this situation.

For instance, tools like Rsync have been around for a long time, not only can it be told to backup only the differences, but it can use various degrees of compression over the network link. Also for those avers of scripts, or the console there are quite a few front ends with a GUI for Rsync.

Another helpful tool is a file system called ZFS, which spots blocks or files that are identical and just keeps ONE copy on the array. I'm sure there are other file systems with this feature out there, but this one comes to mind.

What's more there is backup software that will use the same principle, skipping duplicate blocks or files and just copying the pointers.

I'd surmise that using technology like this a 10TB nightly backup could happen in just a couple of hours, maybe even less.

Medium size company here, our weekly total backup is 2.5Tb, nightly differential is around 800Gb.

The main shortcut we've found is moving as much storage as possible into the backup server, thus avoiding the network bandwidth problem.

It turns out that a Dell PERC6 will quite happily have one array of pricey SAS drives, and another of cheap (consumer grade) 2Tb SATA drives.

Sure, the SATA drives aren't as fast (but still much faster than 1Gb ethernet, so Fast Enough), nor as reliable (well, that's What RAID is for), but they cost us less than £250 for a 5Tb array.

I'm currently moving our network software install share onto it (ie low importance data), which should save us about an hour or so on the weekly full backup.

10Gb ethernet? Maybe in five years.