Time to rewrite DBMS, says Ingres founder

Viva la Revolucion

T-SQL has been the bane of my life. After spending years perfecting skills in functional languages, learning database dos turned my hair grey,

You have one SQL Server. You need to talk to another SQL Server. Let's play a game of match up the 25,000 single quotes and consult the Oracle (no pun intended) whether the engine's going to try and pull back 8 million records before carrying out the where clause.

Surely the time has come when we can eliminate the need to communicate with database servers through single terminated lines of pseudo english.

All your db are belong to Web 2.0?

what i hear is just another trendy Web 2.0 pronouncement, like "expertise is dead, long live the wisdom of crowds!" and "we don't need no steenkeeng personal computers, the cloud provides all that we require!" don't even get me started on Sadville.

good to see the old boys trying to keep up, but one of the major virtues of today's enterprise-grade DBMS is that it offers a good balance between data integrity, performance, redundancy, and availability. when H-whatever has the track record to show that it can do as well or better, the time will come to take them more seriously. at the moment, they sound like the semantic-web guys (is web-centric, SaaS vaporware more whispy and nonexistent than conventional vaporware? what is the sound of one hand not giving a sh*t?).

it's 2008, and Web 2.0 solutions are providing new and interesting hacking exploits every week. until the security model is more mature, i'll stick with a DB that exposes no API to anything but the hardened, SELinux-enforcing server in front of it; and since i'm extra-crusty, no PHP, either, you damn kids...now, get off my lawn, before i set the dogs on ya.

Obviously this man is not involved with web development

If transaction locking and multithreading were removed from databases then many web based applications would suffer for it. We're developing a service that must have transaction locking otherwise if multiple users try to edit the same record at once it would introduce a race condition and all edits except the last one will be overwritten. Sure, we could control this in the application but it's much more efficient (and cleaner) to do this in the db because transaction locking is an optimised and native feature of databases.

Perhaps this guy would consider removing support for nulls too? ;-)

Do these people actually work with databases?

<rant>

I do and I can tell you that one of the worst things I have to deal with is those who think that using a browser front-end is a good idea for a large, multi-session DBMS-based application. Instead of actually looking at whether it enhances the application, they look at the sexy web page and say "ooh yes please", without any regard for its efficiency.

Our once-simple front-end application software (which worked) now requires major rewrites every time the user wants an extra piece of data on screen. Software which was once very flexible has been replaced with 3rd party, Ajax-riddled, IE6-dependent garbage, because someone wanted to look a bit more "with it".

It ain't the database model which is broken. It's the attitude of managers who decide that because something is new, it must somehow be better.

One of our worst issues is performance, not because the database is slow, but because they won't fork out for the hardware required to run their expensive web-based bloatware.

If they think DBMS should be rewritten, they should get on with it, but most DBMS users are not going to fix something which ain't broke, so I hope it's just for fun.

</rant>

(untitled)

While getting rid of transaction locking and so on isn't immediately intuitive, I'm sure there could be some propeller-head way of working around potential issues there.

What I do object to is getting rid of nice simple SQL statements! I'm not a goddamned programmer (I'm not a DBA either), and I hardly see the need for having to learn a whole layer of obscure syntax in order to carry out database procedures. So are we now going to get programmers maintaining databases?

A dba (and some of us pathetic sysadmins) has portable skills - learning the ins and outs of a new database engine is reasonably straightforward (although there will always be Oracle and *sql gurus). I really don't like the idea of yer average programmer being in charge of DB *maintenance*, at least for anything that fits into something larger than a Sqlite or MSDE instance.

This is not new...

Since (at least) the early 90s the research community has been working on looking at breaking out of the paradigm of the traditional DBMS. The way we look at data has changed. With the Internet, data is out there and the DBMS community is struggling to find out how to cope with it. So there are researchers out there trying new things out, such as data appliances. p2pdmbs, MapReduce etc.. Some of these things will live and some will die.

Unless you are at the bleeding edge I wouldn't worry about it until it comes out of a shrink-wrapped box.

Not news...

.. this is already happening in lots of big companies. When your application needs a fast in memory database chances are you've already wrote one yourself. Iyt makes sense to start using this for your reporting databases as well.

Time to rewrite ...

Good article, as is anything that makes people think (and/or Rant).

@David, "Yuk!" - I agree, when and if someone comes up with something better. BTW don't diss my sofa ...

@Trix "(untitled)" - Problem : if it's simple enough for sysadmins to use then it's also simple enough for accountants and engineers to use (from Excel and Access). Therein lies another can of worms ...

@ AC "Maybe it is time ..." - DL/1 ? Pfahhh, surely you mean Pick or even Mapper ?

This guy is a prophet!

As usual, what this guy writes makes an awful lot of sense,

Nearly all of the features listed are needed by the industry, I'll take an example, data stored in a btree distributed across multiple nodes.

Currently all the applications (e.g. memcache) in this space use a DHT because the technology is well understood and simple.

One problem with DHT, you cannot do range queries. Whilst to some extent range based queries can be avoided this is not always the case.

This is where the future lies (and I'm not going to disagree with anyone who says the SQLXML crowd are a bunch of idiots, probably the same idiots who thought EJB was a good idea).

Storing databases in RAM?

> How do you keep data safe when dealing with system ?

> reboots or server downtime?

> Nothing is ever up 100% of the time.

Easy. For system reboots, simply copy the databases to disk before rebooting and load them into ram again on startup(makes reboots a bit longer, but on a production server they shouldn't happen that often). Or even send them to another machines ram over the network.

For unplanned downtime/crashes etc., work out how much data you are willing to loose. Is the past hours worth ok to loose for an average crash once a year? If so, then back up the ram to disk each hour. This is a task that can run in the background while your box continues its job.

geographically dispersed RAM storage

The overhead of packaging your update up and sending it over the network to a remote system and then waiting for the reply will almost always be greater than the overhead of a well maintained RAID system, especially with a good write-back cache.

Sense prevailing at last?

Most DB accesses are reads, not writes, so optimise for those. As for crashes losing the contents of RAM: You don't need to write the DB to disk, just the transaction log. Just replay the log to recover in the rare event of a crash.

Secondly, the RDBMS didn't take over the market because the relational model was superior to alternatives - just that SQL was vastly superior to the other options in the 80's and 90's as a means of getting value out of the data you had accumulated. The problem with the relational model is that it doesn't reflect a real world abstraction. Sure any problem space can be mapped to a relational solution, but often that mapping is convoluted (hence the object-relational problem that still hasn't been solved).

Network-hierarchical databases are pretty damn close to the ideal, as long as the tools to get at your data are good enough.

Ah, Codasyl

XML - sigh

I read Oracle's early papers on XML DB - "we store the schema and the data separately, with pointers into the data"

Codasyl in disguise. The same with the o-o databases, hierarchical databases - none of this is new at all.

Personally I can write SQL, I understand the syntax and have been using it for over 20 years. Once you realise you're really working with sets it's pretty easy to understand.

RDBMS is now just another commodity. I'm happy to use whatever comes next, I currently develop in Ruby on Rails / MySQL and hardly ever have to see the database directly. Interestingly, the people who designed Active Record made 90% stuff really easy and then allow you to drop into raw SQL for the complex, instead of retrofitting the kitchen sink.

I'll use whatever comes down the pipe, as long as it meets my needs - that's it.

This is really an article about hardware evolution...

...and how it invalidates the assumptions underlying earlier architectures.

Stonebraker is pitching the idea of "one thread, one core" and seeing how far he can go with it. (We all know how far the "one man, one computer" idea went...) If it takes the market a few years to provide enough cores per server to make it really take off, what of it?

It's practically a foregone conclusion that each core will have its own large, dedicated RAM to sidestep the symmetric multiprocessing shared-RAM access bottleneck.

Nothing new under the sun, at least nothing that would surprise a Tandem programmer. The Tandem designers figured out back in the seventies that loosely-coupled, independent processors were the best way to meet their self-imposed fault tolerance and linear scalability requirements. Although multicores on a single die won't be quite as fault tolerant due to physical and electronic proximity, they open up vistas of parallelism and linear scalability unachievable with SMP. The cores will probably use a single-threaded, message-passing microkernel instead of a more general-purpose, multi-tasking message-passing microkernel like Tandem did, and I think Stonebraker has assumed as much.

With continuing obscene increases in RAM, a single core could be tasked with handling standalone, in-memory manipulations of a subset of your tables, the centerpiece of Stonebraker's argument and H-Store design.

The late Jim Gray gets a lot of credit for bringing the sacred fire of true database religion from UC Berkeley to Tandem, where Stonebraker's H-Store has already been prefigured in the Tandem Disk Process. The Disk Process handles SQL queries meeting certain simplicity criteria for tables on the disk that it manages. The substitution of RAM for disk is already a commonplace, so where the H-Store breaks new ground is in giving each H-Store it's own processor. The infrastructure will need just enough kernel to manage memory and message passing (the H-Store application being single threaded) with the result that it will fit into a multicore core buffed out with enough RAM.

This is why most of the paper seems to me like a respec'ing of the Tandem Disk Process for multicores, with a rethinking and reallocation of responsibility for fault tolerance, not an abandonment of the same. Recovery becomes more coarse-grained simply because the computing elements are so much more powerful than they used to be.