back to article MySQL's growing NoSQL problem

Just a few short years ago, MySQL was the undisputed king of the open-source database hill. But with the NoSQL market emerging at an 82 per cent compound annual growth rate (CAGR), it's looking like MySQL may get bulldozed by its NoSQL peers. While this shift toward NoSQL provides an interesting commentary on where the …

COMMENTS

This topic is closed for new posts.
  1. Notas Badoff
    Pint

    No zero-sum brawl here

    "What isn't healthy in that chart, however, is the rise of the NoSQL database market."

    Somewhere in all this good news you lost me. I'm thinking you somehow were meaning to say mySQL now has competition, and that that's bad for mySQL. Then you show me a chart that says they quadruple revenue in 4 years. I'm supposed to shed tears because that could'a been quintuple instead?

    Then you quote two sources two years apart that show markets explode in unexpected directions, and try to quote an estimate for 3 years from now?

    Sounds like all good news. I can remain a fuddy-duddy bigSQL user while others can be castor-oil powered nanonuSQL users. There's choices we never imagined, and mores'a'coming. Think of all the intermural developer slagging possible. Fun fun fun!

    BTW: Surely it's okay to have an article cover two interrelated subjects, without pretending it is just one overheated subject. "Outlook for MySQL and noSQL" is the subject here, not "Orcs rule, goblins drool!"

  2. Anonymous Coward
    Anonymous Coward

    Insert pithy witticism here

    Why would people pick MySQL in the first place? Because "everybody else does it", and they don't know any better. As the M in LAMP it sees a lot of automatic deployment even if a different approach might've been a better idea. It happens.

    Why would people move to NoSQL? Because it's the hype du jour, or they (think they) need "web scale" or other applications that if you would try to tackle them with "traditional SQL" you risk running head first into a brick wall of conflicting requirements. Or maybe because you can get better performance with comparable guarantees if you're willing to put up with the fiddlyness that's inevitably part and parcel of the relative immaturity of the new field.

    Why would people move from MySQL to PostgreSQL? Because they realised they really do need guarantees like ACID, which MySQL only "follows"*, not actually really completely provides. And half a guarantee isn't a guarantee, even if most casual use is almost certain never to run into the corner cases where MySQL drops the ball.

    So the field is changing. Is this a bad thing? By no means. MySQL is a nice hobbyist tool with bells and whistles and things. It serves lots of basic tasks pretty well. It even scales to a certain degree, thanks to lots and lots of people pushing it onward. But if you need something better, well, you need something better.

    I disagree it's getting harder to choose, actually. You can still pick MySQL, or anything else you fancy, and hope to the skies above that it's good enough. Since the field tends to improvement the chances of picking something that'll be reasonable close to good enough are continually improving. And if that doesn't work you can (nearly) always throw more hardware at the problem until it goes away, and that happens often enough too.

    The way to get a really good fit is still the same as before: Understand the problem, the sort of queries you do on what sort of data and so on, understand the theory of operation, and let your choice be driven by your requirements. With ever more options, the chances are there'll be one or a few "close enough" and maybe some will be open source too.

    The problem then is more that you have to understand enough of the various options to make an informed choice. Since most of the NoSQL wave are more or less niche products, you can ignore them if you're in a different niche. That then leaves the more generic ones to choose from, and if you are informed you can make an informed choice.

    As a trendwatching piece the article is a tad pathetic. CIOs shouldn't worry about this sort of thing. They should instead be hiring people who bring the right know-how to their projects and who'll pick the right tool for the job. Which, short of projects that are indeed best served by yet another LAMP stack, rarely would be MySQL. But then, there's still CIOs that hire MCSEs and A+ certificees, so MySQL will have a future for quite a while yet. No, MySQL was never my first choice, as you probably can tell. Popularity alone doesn't get the job done, and when all's said and done, it's the work that counts.

    * That's what the manual says. It also says that constraints clauses are parsed but ignored. That chucks the C in ACID right out the window. If you need constraints at all, you need something that can provide that functionality reliably, not just when it feels like it.

    1. Destroy All Monsters Silver badge
      Windows

      Re: Insert pithy witticism here

      "It also says that constraints clauses are parsed but ignored."

      LOLNO. MySQL sometimes behaves like a pregnant camel with gout but not here.

      1. Anonymous Coward
        Anonymous Coward

        Re: LOLNO

        "InnoDB tables support checking of foreign key constraints. [...] For other storage engines, MySQL Server parses and ignores the FOREIGN KEY and REFERENCES syntax in CREATE TABLE statements. The CHECK clause is parsed but ignored by all storage engines."

        Sayeth MySQL. So yes, it does, and yes, that's consistency right out the window, as documented in the manual. Which is better than no real consistency but not documented, I suppose, but still. Note that unless using InnoDB you don't even get referential integrity. And even if you do, you get something that's, er, quite easy to break, see the "important" section below the quoted section in the source. What do you mean, relational database management system?

        It may be worth remembering that the point of having a RDBMS is that it provides not merely structured data access, but also some very hard guarantees (those pesky ACID ones) that otherwise you'd have to reinvent yourself. Which is why you really shouldn't go pick database software nor designing database layouts without passing an introductory course on database management. But hey, as long as you have recent backups it's all good, right? Right?

        1. Anonymous Electronic Warfare
          FAIL

          @Robert Long 1

          I am far from being an academics person - I see myself as a software/systems engineer. I am working for businesses who have real money to lose if their valuable data gets corrupted. I feel really shitty if I have to run "repair table" on a MyISAM table filled with expensively generated data. And I feel even shittier if the number of rows is smaller than before the "repair" action.

          In one of my last jobs we had a 60GB MyISAM DB and I felt like being very smelly and brown in that job. We did nothing exotic and had only minimal query load (less than 10 queries/s).

          I assume all your consulting jobs are department-level DBs which can basically be burned down without much damage. Or worse, you don't realise the problem.

          So from an engineering point of view, Postgres is in many (integrity, query optimization, feature set) ways superior to MySQL. And that translates in much better economics on the long run, as your trousers won't be pulled because MySQL fscked up your mission-critical data. There is a reason some people even spend lots of money for the Ora RDBMS. They want insurance against Business Destruction.

    2. Anonymous Electronic Warfare
      Stop

      MyISAM WILL shove a rod into your anus

      It is just a matter of time until MyISAM will destroy a sizable portion of some critical data if you are so stupid to use it.

      Innodb might be better, but the very fact that they promoted MyISAM means the MySQL people are simply Marketing Muppets.

      Real developers chose Postgres or OpenIngres. Or the slavery of Larry, which is still better than MyISAM.

    3. Anonymous Coward
      Anonymous Coward

      Re: Insert pithy witticism here

      "Why would people pick MySQL in the first place? "

      Because it's good enough. That's the bar you have to reach, and it reaches it. I've maybe 50 clients using MySql (and one bastard on MSSQL), many of them major companies with 60,000+ users of the systems in question. If MySQL works, why NOT use it?

      ACID is not as important in the real world as it is in academia. It's not that ACID is unimportant, it's that there are other factors and it is only in the realm of "hobbyist tools" that one has the luxury to pretend that cost, ease of recruitment, speed, and support are not important.

      PostgreSQL has suffered for decades with the problem that its developers are dyed in the wool theorists and its supporters create a user-hostile atmosphere for anyone who tries to inject any note or hint of reality into the theory. That's why it's STILL obscure after all this time. I have literally never seen a PostgreSQL install outside of universities. Not one. But the problem isn't with the DB.

      Anyway, after watching Oracle's behaviour over the last few years, I'm migrating to MariaDB anyway.

      1. Phil Endecott

        Re: Insert pithy witticism here

        > I have literally never seen a PostgreSQL install outside of universities. Not one.

        http://www.postgresql.org/about/users/

        Just to pick a few from the long list on that page:

        IMDB

        Safeway

        Skype

        Apple

        There is also a list of users of PostGIS, the geographic version of PostgreSQL, here:

        http://postgis.refractions.net/documentation/casestudies/

      2. Anonymous Coward
        Anonymous Coward

        Re: Insert pithy witticism here

        MySQL is only good enough if you don't care about consistent performance (thanks to a shitty optimiser, table level locking and inneficient joins on MyISAM tables), can tolerate corruption of data with something as simple as reindexing, intuitive behaviour (for example extend a column type, say int to bigint, and unlike most RDBMS any index using that column will still treat it as an int), and downright bullshit in the documentation (integrity is only ever "an application level concern", which they claimed since MyISAM doesn't support foreign key checking). That's just the tip of the iceberg. InnoDB is poorly documented (try performance tuning it), and can drop indexes entirely without warning - a bug that bit me.

        As for PostgreSQL, it's used all over the place, which is why it supports a growing number of large consultancies and companies selling services built on it. It hasn't been a primarily academic projct since the mid-90's, and sorry, you show your ignorance if you think that the features PostgreSQL supports that either MySQL doesn't or only does so poorly are niche ones. For example. PostgreSQL has a mature, performant and very well featured full text indexing plugin. MySQL in comparison has a piss poor full text indexing feature (a single compiled in stop word list for example) which only works on MyISAM tables, not the InnoDB ones you use if you care anything about your data.

        As for switching from MySQL to Monty's hooby project MariaDB - good luck with that. At least Oracle have competent developers.

      3. This post has been deleted by its author

      4. the spectacularly refined chap

        Re: Insert pithy witticism here

        I agree with Phil here. If you've never seen PGSQL in the wild that says more about your employers or customer base than the virtues of one DB or another. It seems to me that MySQL is and alwys will be regarded as little more than a toy. It seems to have got a little better over the years but not much, and from a comparatively low base.

        PGSQL is a much more fully featured system in comparison. These are not theoretical concerns - there's a reasonable sized PGSQL DB here that absolutely depends on deferrable constraints (because of circular references) for example. Good luck implementing those on MySQL. You can't, and the first time you notice that you have a problem through the lack of those constraints it's already got to the point it would cost a fortune to fix.

        I suspect the reason MySQL got so much share in the first instance is because it is cheap (computationally, not in terms of financial outlay). A lot of web hosting deals throw in MySQL as a freebie - a stronger DB attracts a hefty premium.

        1. Anonymous Coward
          Anonymous Coward

          Re: Insert pithy witticism here

          "I agree with Phil here. If you've never seen PGSQL in the wild that says more about your employers or customer base than the virtues of one DB or another."

          Sure. The point is that you can go for years in this business and not encounter the need for what PGSQL offers. There certainly comes a point when you do want them but there's an almost religious belief that every small company or university needs them more than it needs anything else. MySQL is cheap, both in terms of computation and financial outlay, and the fact is that that matters. There is a lot of useful support available for it online; that matters too. The people who use it don't have a chip on their shoulders, and that matters a bit too.

          The question I was responding to was "why does anyone use MySQL?" And the answer still is "because it's good enough for what they're doing".

          If you want to change the question to "why does anyone use MySQL and then complain about it not having feature x, y, or z" then that's a different question.

    4. Charlie Clark Silver badge

      Re: Yes, but..

      Why would people pick MySQL in the first place? Because "everybody else does it", and they don't know any better. As the M in LAMP it sees a lot of automatic deployment even if a different approach might've been a better idea. It happens.

      10 to 15 years ago MySQL was also available for Windows users which meant that many new developers chose it as they could run it on their own computers. As with so many other things at the time this meant that impressionable people chose tools because they were easy to use rather than their suitability for the job. That was bad enough, but it was compounded by the abundance of poor advice and hacks to compensate for deficiencies of said tools. SQL injection would never have been a problem if code examples and libraries demonstrated the use of pararmeterised queries.

      Much as I personally detest the MyASM backend it was developed for a reason. I just wish this wasn't the default behaviour as errors in your application are likely to be exposed after deployment. Data integrity is fucking difficult to impose in the application layer which makes it one of the main reasons for using a data management system. You can get the pretty much the same performance using an ACID system if you selectively switch off or defer constraint checking but this is like two consenting adults agreeing to have sex without a condom - they are aware of the risks.

      Of course, MyASM is supposed to be a great funnel for customers to upgrade their applications to Oracle once the failures inherent in a non-ACID data-based application start to appear.

  3. Justicesays
    FAIL

    Maybe Some Bias

    ". If you take a look at market share in the server space, as you look at traditional data centers, about 70 per cent are running on the Windows platform and about 30 per cent are running Linux."

    Man, UNIX is doing worse than I thought !

    http://www.theregister.co.uk/2011/07/11/gabriel_unix_server_survey/

    1. Anonymous Electronic Warfare
      FAIL

      Bullcrap

      "70 per cent are running on the Windows platform and about 30 per cent are running Linux."

      Most data still resides on MVS mainframes, especially the important stuff. An yes, there is HPUX, AIX, Solaris and a lot more which host huge databases.

  4. Anonymous Coward
    Anonymous Coward

    No means no

    It doesn't make sense to imply long term trends based on these short-term trends. What is happening is that there are now better tools for handling non-relational data so software like MySQL stops being the first choice for stuffing everything on disks. Anyone who thinks that relational data is dead is in for a fatal struggle against reality. Those who can chose the right tool for their data are winning with the new options available.

    1. Destroy All Monsters Silver badge
      Pint

      Re: No means no

      Quite so.

      It's all like there is a new barber in town.

      What happened to the old BerkeleyDB btw?

      1. Phil Endecott

        Re: No means no

        > What happened to the old BerkeleyDB btw?

        Well of course it's still around, but I think that domain is increasingly moving the SQLite.

        1. Anonymous Coward
          Anonymous Coward

          Oracle bought sleepycat software, remember?

          SQLite is a bit of the developers darling, lately. "Oh lookit, it's just this small library and it gets you SQL and shit, yo!" -- And certainly it's relatively small, and it does do half-decent SQL, too. Only half-decent because it doesn't enforce types in columns so you can stick an int among the strings and a string amon the ints, and such. But at least it does better referential integrity and triggers than does MySQL, so in that sense it's preferrable. Compared to that other hype, XML, I think I might prefer them devs to stick to SQLite, even. SQL, even without columnal type consistency, is wonderfully standardised and a lot more powerful compared to the various XML query languages, actually usable even.

          But as a replacement for key/value store, uhm. SQL is about relations, key/value is not, other than finding again some item stored earlier, and be quick about it. You can do that in SQL, certainly, but you're leaving a lot of performance on the table going that route, as you wouldn't be making use of SQL's strengths.

          1. Tim Almond
            Boffin

            Re: Oracle bought sleepycat software, remember?

            XML isn't hype. If you're using it for storage, you're probably doing the wrong thing (although it's quite good for config settings), but as a way to move data from system to system, there's nothing better.

  5. Jeff 11

    The fact is that there's more (valuable) data out there that benefits from structure and the rich query language of SQL, than unstructured data that can benefit from the performance and scale of key-value stores. If you're talking about figures and metrics, then you need SQL - or rarely, something even more specialised - and your average profit-making business generally needs figures and metrics more than it does document stores. There are academic and big data exceptions, of course, but they're a drop in the ocean of global installations.

  6. Peter 39

    actually, no.

    "That's an amazingly fast shift, and it says a great deal about how open source drives innovation."

    Actually, no. It says a great deal about how Oracle ownership drives open-source innovation.

  7. Tom 79

    Traditionalist

    I'm a traditionalist in the fact that I've been using RDBMS (Oracle and recently MSSQL) for years now. Would someone explain why NoSql is better than a simple key/value table with an index on the key in a traditional RDBMS from a purely technical perspective?

    1. Novex

      Re: Traditionalist

      I also have difficulty understanding it, so I went off and read up the wikipedia entry here http://en.wikipedia.org/wiki/NoSQL, with the obvious caution that is, after all, a wikipedia article.

      From what I can gather, the main reasons for ditching an RDBMS are the CPU/Memory overhead involved in running said RDBMS (all the stuff that keeps data consistent, etc) for very simple data structures, and the problems involved in distributing a 'single' database across multiple systems where the amount of data is too large for any one system to handle.

      The biggest drawback of NoSQL (as far as I can tell from the wiki) is the sheer number of different NoSQL database systems out there. There seem to be so many versions from different vendors, each with its own API that it would be pretty much impossible to know them all as a database developer. Each variation will have been devised to optimize to the extreme one form of data storage and retrieval such that no other system could hope to be as fast (which may be a good thing for fast, lean systems), but the problems to the development community of the fragmentation these NoSQL systems cause, and the headaches in supporting them all in the long run, do worry me.

      1. Oninoshiko
        Boffin

        Re: Traditionalist

        you are mistaken on one point.

        the biggest drawback of NoSQL systems (and lets be honest here, Object stores and Document Stores atleast have been around a while and are similar enough that they suffer from the same problem) is the lack of gerrentees on data relationship consistency. This may not be a problem for trivially simple structures, but the moment you have even one relationship between datums, this is an issue.

        Wildly differing APIs is a DISTANT second.

        (other then that, you are correct.)

  8. Tom 7

    Not a mention of Oracle/Google battle over Java.

    If you think MySQL is in safe hands then you need to check out Groklaw on this one.

    As for NoSQL I have a feeling that the idea behind it that complicated is somehow bad. Sorry, its called reality and its not bad, you cant avoid it and in the long run you HAVE to address it.

    You can, as with computer based 'offices' of many sources, avoid the issue and continue to de-compress and de-organise data in a million 'easy' forms rather than pop it neatly into the filing cabinet but some day the toy cupboard will have to be tidied and it will all spill out on the floor and you may realise that had you done this a lot sooner several very large companies would not have had a very large cut of your revenue.

  9. Sean Timarco Baggaley
    Thumb Down

    Matt Asay's bias is showing.

    "Open Source" is mentioned 12 times in the article, as if it had anything to do with the price of fish. It's a product distribution option, nothing more. Claiming that "Open Source" in some way stokes "innovation" (an assertion made in the article) is just as big a load of bollocks as claiming boxed software stokes innovation instead.

    People use tools in order to do a job. Whether the tool comes with a maintenance and customisation guide is irrelevant given how very, very few customers will ever bother to read it, let alone act upon it. The vast majority of Open Source software users just install the damned package and leave it at that. That the source code to each package is available to download and build makes no difference whatsoever as most wouldn't know what to do with it anyway.

    1. Anonymous Electronic Warfare
      Stop

      Re: Matt Asay's bias is showing.

      Linux, BSD, XML, HTML, Qt, wxWidgets, Apache,SVN, WebKit, TeX, rsync, Android, Eclipse. It can indeed be argued that most of the innovations of the last ten years have been driven by open source.

      Commercialware vendors haven't done much innovative recently. Their contraptions are just nicely polished Balls Of Shit. Scratch the surface and you will smell it. Also, these balls are 100 times larger than technically required.

      Just take Chrome, iOS and Android - they would have never happened if Google and Apple would have had to develop all from scratch. Meanwhile M$ is struggling under the load of their own contraptions - just fixing the bugs is too much for them. What they manage is to polish their stuff in randomly new ways and rearrange everything like a department store. But there is precious little innovation.

      Windows 7 still has the same old stinking pile of SMB (they now call if CIFS ?) under the hood and it is still making the same difficult-to-reproduce error we already had in NT 4. They still don't understand the difference between root and normal user - UAC is a big fat abomination. IE 8 is a fat snail while Chrome drives the innovation. M$ was forced by Chrome into standard JS/HTML5 and to abandon Silverlight.

      So, yes the economics of open source are very well visible if you look at Google, Apple, Redhat, Facebook and many others. And yes, the economics make room for innovation, while the proprietary vendors are stuck in their legacy stuff (IE, NT kernel, Office etc).

    2. Rampant Spaniel
      Flame

      Re: Matt Asay's bias is showing.

      His recent work has declined significantly in quality. His GM \ Facebook article was terrible, and we never did get an answer if he has shares in Facebook.

  10. Charlie Clark Silver badge

    I'm intrigued

    Another predictably poor piece of OpEd from Mr Asay.

    I am, however, intrigued by the silence around the third category in the soothsayer's diagram: NewSQL. Is this an attempt to remedy the inherent deficiencies in the language? Or perhaps a set of engines that successfully distinguish between the logical and implementation layers?

  11. Ken Hagan Gold badge

    Is this innovation, or just diversity?

    Are these products genuinely different, or are they merely "pretty much the same code, performing the same end-user functions, under a different licence"?

    The billion and one different Linux distros aren't innovation. The choice of LibreOffice versus OpenOffice isn't innovation. Even comparing the JVM and CLR, which are normally thought of as different ecosystems, I see basically just a clone of the basic features of a runtime environment as pioneered by ... well probably someone in the 1970s that I haven't heard of.

    For me, when someone starts talking about "innovation", unless they immediately provide pretty startling evidence of novelty, I just assume that it's marketing/management bollocks. Sorry, but that's a 99% effective filter and it is really easy to apply.

    1. Anonymous Electronic Warfare
      Go

      Re: Is this innovation, or just diversity?

      I do indeed think that commercial vendors have not managed to create a proper webserver, a proper version control system, a proper web browser with fast JS and good sandbox, a nice GUI toolkit during the last 10 years on their own. All the "good" stuff has been full or partial open source. Yes, Chrome is innovation - try IE 8 if you don't believe me. Check the list of exploitable bugs in Chrome and FF if you don't believe me. SVN and CVS are much, much better than all the commercialware contraptions, because they simply work instead of presenting stupid metaphors and half-broken tools. That's innovative.

      Ubuntu is innovative because patching is a single-click affair. Linux invented this, everybody else followed.

      Apache plugins are innovative and it's stability and robustness is innovative. Just check IIS.

      Gcc is innovative because it generates for everybody's and their dog's instruction set.

      HTML is innovative because it is not a property of Adobe, Microsoft or Quark.

      1. Anonymous Coward
        Anonymous Coward

        That's not what innovation means.

        Innovation simply means "bringing something to market that wasn't there before".

        This can be an invention (something truly new), but innovation doesn't have to be technology. It can be as simple as selling the same old with a new financing plan, something that has occasionally driven up sales. So yes, innovation easily is just a bit of marketeering, but no less innovative for it.

        Companies all assert they are "innovative" because if they weren't innovating they'd be stagnating, therefore doubleplus uncool as companies, and so on. But just asserting it a lot doesn't guarantee their innovations really are innovations, or just that much smoke and drivel. And even if what they did was truly innovative then that still wouldn't guarantee their offerings were any good. That just isn't part of the "innovative" yardstick.

        Most of the examples given really aren't "innovative". This is not bad by any means. In fact, open source can get by without having to be seen as a cool company. All it needs to do is do what it says on the tin, and do it well. Well, that, and have a thriving community to keep the thing from bit rot, fix bugs and security issues, and all that. But that's hardly innovative. Releasing with new features might be, but too many of those and you get bloat, perhaps dysfunction, and you risk project forks and so on. But I digress.

        The point is that the reasoning why these examples would be innovative largely has nothing to do with innovation at all. This isn't bad. It just isn't innovation.

  12. Chris_Maresca
    WTF?

    Meanwhile, in the real world, companies are abandoning NoSQL

    and migrating to MySQL

    http://saucelabs.com/blog/index.php/2012/05/goodbye-couchdb/

    Basically due to scalability & stability problems...

  13. Antony Riley
    Thumb Down

    Seriously.

    Why is it news that a product in a new market segment shows more growth than a product in an old crowded market segment.

    1. TheOtherHobbes

      Re: Seriously.

      To most people it's not.

      To Matt Asay - the penguin only knows how this man's mind works.

      I've been trying to make sense of it for months now, and all I can see is a list of faddy brain spasms and wannabe-corporate non-sequiturs.

  14. Sirius Lee
    FAIL

    Content free article

    There's a place for NoSQL but not as a replacement for a Full SQL product. Whenever you need to join data across two tables NoSQL has no place. If you need to change the schema or have lots of 'variant' data and it's not worth creating a mechanism to handle these in discrete sets NoSQL is a solution.

    The canonical application for NoSQL is as the Craigslist backup. The team maintains a moderately large database of current items. It must perform. After a while aged records are dumped to a much, much larger archive database. When the schema changes to accommodate new ideas, products, etc. its necessary to update the schema of the current items database. But what about the archive? It's huge and changing the schema took weeks. Enter NoSQL.

    This is a good use of NoSQL. However, it's got to be a concern that kids are going to be creating transaction systems using NoSQL. That would be a disaster.

    1. Semaj
      Pint

      Re: Content free article

      "However, it's got to be a concern that kids are going to be creating transaction systems using NoSQL. That would be a disaster."

      Indeed it would - and it is happening. In no small part because of articles like this for the reasons already stated many times above.

  15. PeteA
    WTF?

    NewSQL

    Survey submitted

    Your input has been registered.

    Thanks a lot for filling out the survey!

    Results of the survey will be available about in August 2003. They will be published on this site.

    Click here to go to the main page.

  16. dwight

    it's not really about MySQL per se

    fwiw, I see more Oracle->mongo migrations than mysql->mongo.

This topic is closed for new posts.

Other stories you might like