Generally, it seems to me that object storage is suffering from a failure to launch despite more than a dozen suppliers pushing it. Many of these same vendors seem to have their heads in the sand with regard to their place in the marketplace - they seem to ignore the fact that end-user buyers are confused about what object- …
"end-user buyers are confused about what object-storage really is"
Damn right. Even the Wikipedia entry on the subject is woefully vague. So come on, El Reg, tell us what it is?
1. No standard - er, CDMI?
Sorry to say it - this is a pretty scrappy article. Not well thought out.
You leap about between discussing object stores, and then comparing them with the underlying technology - you mention 'tape' several times. The actual mechanism for storing the data is separate from the data.
I was at a talk recently by DDN and was very impressed by their Web Object Store.
Similarly a lightning talk by Hitachi at Cloudcamp on object stores. It is an idea whose time has come.
The problem with "object storage" offerings is that they are all based on slow and bloated subroutine libraries intermediating data access, reducing performance to a crawl. Nobody wants that.
They are also based on opaque proprietary standards that transforms an otherwise-normal data storage system into a data prison you can never leave (or backup). No one needs that either.
Storage California - you can Check Out but you can Never Leave...
Regarding your point about backups, you don;t back up huge amounts of data in a big store like that.
You keep two (or more) copies, hopefully in distinct locations.
Sorry if that is me appearing to be aggressive - I run an HSM system, and backups consist of backing up the 'stub' files on disk. you then keep two copies of the data on the slower tiers.
Object storage...not so problematic
I suspect the article was written to generate more heat than light on the subject, but storage vendors do have an industry association in SNIA and SNIA is driving the CDMI standard into the storage market at a pretty good pace when it comes to standards. That said, AWS S3 is the de facto standard API for object storage and CDMI will be able to work in conjunction with it.. I spent the better part of last year evaluating a handful of vendors who provide object storage software to enterprise customers and partners to build private and public storage clusters. Each of the vendors is venture backed and their founders all have significant history in dealing with data storage requirements that were not solvable with the traditional file and block storage technology that we've had around for decades. The incumbent storage vendors have taken out their checkbooks and bought the storage technology they think will allow them to participate at-scale in the market. Whether they can be price competitive remains to be seen given their past history of bundling their storage software with their proprietary hardware. Because the object storage market is relatively new, you can expect to see some participants get acquired and some incumbents to change course. The recent Dell announcement about ending their OEM deal with Caringo has more to do with the internal players at Dell than it does with the quality of CAStor. All of this is par for the course and it may be quite a few more years before the object storage market and players coalesce. Remember that there were once over 200 vendors engaged in the manufacture of hard disk drives. After 30+ years we have 4 of them left. The same thing is going to happen in the SSD market too. There is a market for object storage and it is being met by offerings from relative new companies as well as the incumbents. It is not a matter of a technology in search of problem.
Re: Object storage...not so problematic
@cloudguy: I should take heed of someone who tries to explain a modern technology but has no grasp of the old-school concept of readability?
Ever hear of a paragraph break, cloudguy?
Article FAILS to launch
Chris, this is a hack job of an article and it reflects poorly on you.
"IBM - nothing, zip, nada except maybe vague hints of GPFS having object connections " - REALLY?? You couldn't have googled "IBM Object Storage" and looked at the copious Product, Research and information? Let's have some basic research completed before putting mis-truths out there.
The net of the problem is simple: Object storage, when properly used will require application (re-)design changes. I'm fond of the phrase: "the only human that likes change is a baby with a dirty diaper". 99% of software developers are used to working with a file system and don't know how to design an app using object storage or better.. don't care to get off that really comfy broken (in) file system. This creates a natural barrier for mass adoption (which according to this article is a failure on vendors part to put out object storage). Instead, developers would rather go to great lengths to do things like write algorithms to generate long, random file names for uniqueness (so there's no collisions and over-writing failures) for things like log files. Dealing with OS and file system differences generates bloated and brittle code. Why wouldn't someone want to reference storage via a URI?
Object storage is a fine technology (/religion?) with the promise of something great. I find it odd and interesting that OpenStack and Swift aren't called out in this article.
Come on El Reg... if you're going to take on a topic, do it right and stop the vendor bashing and focus in on the problem.
Re: Article FAILS to launch
As a developer I can't honestly claim that I've had hassle with porting to account for OS & file system differences very often in recent decades. I've had trouble getting anything like reasonable performance with file I/O under Windows, but that's a problem with Windows rather than file systems as a whole.
Given that the languages and their runtimes have built in support for file systems and *not* object storage I don't see the situation changing anytime soon, and quite frankly for the majority of jobs out there I see little or no advantage in writing code that tickles object storage instead of file systems.
Newsflash: the unique filename problem was addressed by mkstemp(3) which has been around since 4.3BSD, nearly 27 years ago. It's not the worst case of NIH I've seen though, nearly everywhere I've worked reinvented syslog over and over and over and over and over again.
F*ck you, I'm an Object Store!
Doesn't OS/400 have one of those?
The price ain't right
Object storage's (OS) problem is NAS is pretty well freeware except for NetApp. Why pay so much for OS when free NAS does much the same job.
However, OS can scale much better, and searching is much easier, so it fits with S3 and so on really well, but for run-of-the-mill work, NFS is good enough, though it doesn't protect data too well and has no guarantee of availability.
One huge piece of the equation missing in the article is the freeware (open-source for purists) now coming to market - Ceph, and for a small amount of cash, RedHat Storage server. They solve the NAS versus OS conundrum, but they also beg the question of why NAS and block-IO continue to separately exist, since their object store bases provide all three service formats.
lack of demand
I think a lot of it is lack of demand. There is demand of course in large service providers -- those places often have resources to do a lot of things on their own and not rely heavily on enterprise company products. The sheer customer count that needs object storage I believe at this time is quite low (those that feel a need to have it in house). Hence not having much adoption of any of the enterprise object storage technologies.
My company for example uses some S3 for things - but total usage is overall quite small and doesn't justify the need to have object storage in house. Really I think until your into some decent scale with I'd wager at least half a PB of data projected to be stored would folks really get into looking at object storage vs regular NAS which of course is far simpler to manage.
Also don't forget Red Hat storage - from a support and tech standpoint they seem to have a pretty decent and flexible offering. I would not use it as a file server or as a transactional storage system as their marketing sometimes try to convince you to do but as an object system it seems ok.
Just because you can theoretically build a multi-petabyte cluster to store hundreds of nodes' worth of data doesn't necessarily mean you should. Object storage is and probably always will be a niche product, but there are numerous advantages to not putting massive amounts of unstructured data on a filesystem; the first, obviously, being that you don't have to worry about designing and maintaining an artificial structure around it as your data pool grows. You don't have to worry about errant processes on your storage boxes doing things they shouldn't on data being stored as regular files. You don't have to worry about the intricacies of the underlying filesystem, or whether you'll be able to grow your clustered filesystem indefinitely while maintaining scale. You don't have to worry about whether a software bug in a very complex system like Gluster will cause an unrecoverable error or split brain, because object storage tends to be fundamentally simpler than layered filesystems and storage protocols working together to produce the same result.
- Boffins attempt to prove the UNIVERSE IS JUST A HOLOGRAM
- China building SUPERSONIC SUBMARINE that travels in a BUBBLE
- Review Raspberry Pi B+: PHWOAR, get a load of those pins
- That 8TB Seagate MONSTER? It's HERE... (You'll have to squint, 'cos there are no specs)
- Review Reg man looks through a Glass, darkly: Google's toy ploy or killer tech specs?