Judging by my inbox, quite a few businesses have taken to heart my warnings about the legal issues that arise when you allow your data to be exposed to US jurisdiction. Companies outside the US sense a gap in the market and are pouring in. Within a few years I suspect I will be perfectly comfortable with recommending nation- …
what it's worth to me
(as a system admin too...)
Is buying a server and shoving it in a local colocation with 100mbit of unlimited bandwidth on a well connected backbone (about 6ms ping from my local company office, 20ms from home). Server ~$3k, colo ~$200/mo.
Though it provides more than the 3.6TB of usable storage(4x2TB SAS disks hardware RAID 10), there are a half dozen VMs providing email, web hosting, DNS, blog, cloud(owncloud which I am new to), site to site VPN as a backdoor into my home network etc. Add all the services up, and I save tons of $$ over using cloud. Even the storage cost alone blows the doors off cloud storage.
Been hosting my own stuff going back to 1997 or so. Initially started as part of a small ISP, then the ISP mostly folded and the stuff came back home(part of a very geeky home network at the time), hosted on a 1Mbit DSL line w/8 static IPs for several years. At one point I had a 2nd 1Mbit DSL line installed for a friend's non profit org (~10 years ago) to run their website off a server at my apartment.
Finally DSL ISP started giving trouble so I tossed an old server up at a local co-lo in Seattle with a ~2Mbit connection for $90/mo, ran that for a year or so, server started having issues so moved to Terremark Vcloud for a year(~$180/mo with a small fraction of the capacity that I have today).
Then decided to bite the bullet and buy a new server and throw it up at the server vendor's colo for ~$100/mo. A year later the server vendor decided to get out of the hosting game, so I opted to stay in the same facility though had to go direct with the facility and the cost doubled to $200/mo.
Still worth it in the grand scheme of things though. I've had to replace a couple bad HDDs, a failing USB memory stick(which runs vSphere). Other than that it's been pretty painless over the past couple years. Of course anyone doing something like this I urge you to choose a local facility so you have easy access to the system.
I get a full third of a rack, though too power limited to make use of it, my $200/mo gets me I think 200W of power, or is it 240W I forget. I'm tempted in the not-too-distant-future to replace my server with something with a ton more storage.
Back when I was in the colo market in Seattle there was no similar facility to what I am in now (in the Bay Area). There are several facilities here that offer large bandwidth for low cost. I don't need a lot of bandwidth but it's nice to have it available.
The one thing I wish I had is more upload bandwidth on my broadband connection - I currently tap out at around 1.5Mbps upload. The best plan offered is "up to" 5Mbps upload (up to 150Mbps download). I'd love it if I could get bi-directional 25Mbps for a reasonable price (I'd pay up to maybe $200/mo for that). But the only way to get that here is to go commercial, and probably $1k/mo+
Re: what it's worth to me
another upside to local colo is I have the option of sneakernet. I haven't done this yet, but if I want to upload a massive amount of data I can always take a USB HD to the colo and plug it into the server. It's about a 45 minute drive each way, but I can copy data in a few hours what would take me literally months on broadband.
Re: what it's worth to me
sorry one more comment. I did do this once (sneakernet) when I first bought the server, I brought it home, copied ~1.5TB of data onto it and then brought it back to the colo. since then I just do throttled uploads (mostly media ripped dvds/blurays) so that it doesn't interfere with my regular internet activities (porn).
Re: what it's worth to me
Thanks for the write up, Nate. You and I think a lot alike. The primary reason for keeping good backups is the ability to recover your data after something bad happens. If the bad thing is that you just deleted a file or 10 by mistake, having your data in the cloud can bail you out. If you lose a hard drive loaded with current working files, it can take ages to download your backups at the expense of any other access you may need to the internet. Working with a local host with whom you have a personal connection with can make all of the difference with getting back up and running quickly.
My metric is always "Can I throw money at the problem to get it fixed". I'd rather not have to spend money to fix something I have been investing in to save me money when the fit hits the shan, but with a large anonymous service provider there is absolutely nothing I can do. There is no way I could pay them enough money to get ME back up and running as soon as possible. I have no leverage. With a local provider, I can at least drive over and stare at them until I'm back up. If the problem is further up the line, it's likely that those responsible will be working hard on the problem as it will be impacting a much larger number of people.
Nate's sneakernet for large files is a great one. Most of my stuff is small enough that I can back it up online without a problem. If I get into video production, the files will start getting very large and it might be cheaper and faster to haul a Raid box to a colo'd server for back up. The individual video clips would be easy enough to download, it's the giant mass of data from a day or two of shooting that would be a pain to shift.
The cloud providers are alway touting how great it is to have all of your files everywhere you go. My problem is that I am working all of the time and I never get to "go". I tend to plan ahead on the rare occasion when I have time to "go" so I have all of the music I want to listen to loaded on the iPod, all of my contact data is sync'd on my phone and my laptop is sync'd with my home computer. Anything online is as far aways as the moon since I usually can't even get a connection that feels snappy when retrieving email.
May be unfashionable, just as sneakers, but has great reliability and bandwidth. There is only one thing able to offer better bandwidth - a white van full of tapes.
Total loss of control.
The moment you put something offsite, you immediately lose control of it. It's like the proverbial bank safety deposit box. The media makes these things out as secure and protected and they're not. A bank will give you up just as readily as an American Internet company.
At least in the US, you still retain privacy rights to your stuff so long as it remains in your possession. The moment you let it leave your control, all of this "you need a warrant" stuff evaporates.
Plus there's the usual trust issues of letting someone else manage your stuff.
Re: Total loss of control.
This is a valid point, but one solution is along the lines of Nate's post above you - have your own managed server with encrypted storage that you alone have the key. For storage/backup only you don't even need the physical server to be isolated, as you can encrypt-on-write at the client machine(s).
Of course, that is not going to stop a court order for access, but at least they have to deal with your own country's laws which, in theory, you have a democratic input to. That is very different to any foreign host where you can expect a different treatment even to the locals.
And as Trevor points out, you still need a local copy in case the provider has gone badly wrong or is holding your data hostage with usurious fees to migrate your data to another provider...
Re: Total loss of control.
Hiring your own server gives increased security, but no more control than vanilla cloud. As Nate demonstrates, corporations chop and change. One business decision from them and your data is kicked out on its ar5e.
Cloud as a backup target is okay in a minority of cases. Seeding is the bug-bear, and the solution in Nate's case was living near the vendor. But updates can be bad too. Move or rename a couple of large folder trees and your heading for a big sync. And restores are trouble because the cloud is more likely a sync/mirror that a full multi-generational backup. You want your CV as it appeared in July ? Sorry we have only last week's copy.
So the marketing dept is calling networking "cloud". relax. They will have their moment, then move on to something else.
I'm with Larry Ellison on this one
I'm no fan of Larry Ellision, but I'm with him on this one: The 'cloud' has been with us as a technical notion since X.25 days. We have been 'on the cloud' for decades. The 'cloud' as a recent marketing buzzword is relatively recent and sufficiently noxious that Larry Ellison resisted it even though it is demonstrably a great way to charge more for the same thing.
There is no doubt in my mind that our migration from specific premises into data just 'in the cloud' will CONTINUE as it has for decades. However, in answer to the question posed here, is the convenience of succumbing to marketing speak and shifting premises willy nilly to the network worth it? No. I would be stunned if any sane network person who actually knew about the cloud before the marketeers got hold of it would be planning to pay hefty premiums for marketing hype.
At the level of marketing where this article is poised, the talk of 'moving to the cloud' really means moving *your* money into *their* pockets.
Fact of Life?
"Today, virtually everyone accesses a publicly shared storage repository and downloads updates for everything from their operating system to new firmware for their lightbulbs."
As they did yesterday and they are called servers. There is no public write access. I would wager the providers of data they serve do not even consider them to be storage (i.e. they have no real intention to read data from them).
Calling something people frequently use cloud storage when it isn't makes bullshit not facts of life.
Worst aspect of the cloud... permanence.
You've got it all set up and running sweet, all the finances are done, and then... You have two weeks to do it all over again:
I'll stick with in house, thank you very much!
"If you are so big that bandwidth is cheap and plentiful then cloud storage makes sense"
If you are that big then you will find the cost/GB of these services to be prohibitive. Using S3 prices as a baseline:
* At 50TB you're looking at 8¢/GB/month. That's $48K per year.
You can get 50TB in a 4U box with 24x3TB drives for about $8K one-off cost, perhaps $10K for higher-end kit. Double it for resilience, add power/hosting in two data centres (or one data centre + one office location), stick FreeNAS on, and it's still way cheaper to do it yourself. But this assumes you have a modicom of staff expertise available and can spare their time.
* At 500TB you're looking at 6.5¢/GB/month, which is $390K per year. This may take you a whole rack of kit in two places, and you may need more sophisticated software (Ceph, Glusterfs, Swift) but there's plenty of money left to pay for staff.
Given you can now get 4TB drives, and 4U chassis which can take 48 disks (Google for NR40700), the economics swing even more in favour of DIY for large users.
Re: "If you are so big that bandwidth is cheap and plentiful then cloud storage makes sense"
Oh and that doesn't include data transfer charges. Under Amazon's "Hotel California" pricing model, your data can check out whenever it likes, but it costs 5-12¢/GB to leave.
To download 50TB in a month would cost $4800. (Of course there are data transfer costs involved in hosting too, but you probably have a wider range of price options, like paying a fixed rate for a port with a fixed maximum bandwidth)
Not arguing your point, but Steam is not the issue
Just back up your Steam folder to another disk, reinstall your PC on the new disk and copy the folder back again. Log in, give your password and presto, your entire library is again available and useful.
Disk drives do not stop Steam from working. Changing motherboards do not stop Steam from working. Even upgrading the OS is generally not an issue for Steam. Just run the exe, log in and your files are there, ready to work.
I have changed HDDs, motherboards, video cards, reinstalled XP, installed Win 7 (32-bits) and so on and so forth, and every time I just launch Steam, log in and it works. The one time I had to reinstall my games (not Steam itself, just the games), was when I upgraded from Win 7/32 to Win 7/64.
If you stay on the same OS, you do not need to consider Steam as a liability. It is rather unique in that way.
Another one regarding colos...
While one can own the server and co-locate it, the facility can be compelled to allow the government (Law enforcement, NSA, etc.) to seize the server. Your only data privacy hope at that point is full disk encryption and hope they can't brute force the key. Oh, and you also have to keep paying the colo facility- they can (and do!) put liens on equipment there for unpaid services. (remembers a fun trip at one site to perform just such a task for a company that was far behind in payment, and in fact went bankrupt- the company ended up keeping the gear as payment for services rendered up to the point of contract violation.)
Both Sides, Now
I was away all last week. As soon as I arrived overseas, My Blackberry lost its personality and became a consumer device. I had no mobile email or data, only GSM.
Then the third party Internet provider at my hotel puked and stopped giving out IP addresses for three days. Now I really had no mobile email or data.
Roaming on my Android smartphone cost me €15/MB. I was reminded, poignantly, of precisely how little Google sees fit to cache locally, and how much update chatter it donates on my behalf. Thanks Google, my carrier loves you too.
Was I in Madagascar? Malaysia? Mali? No, friends, I was in Mountain View, CA.
I've looked at clouds from both sides now.