Well if no one else is going to comment ....
It always amuses me that people call them databases, and yes there is a logic behind the name, but it is better to understand that what you want is a datastore.
Now, a flat file is always faster than a database if the data is to be read sequentially, and all things are equal. I did read an amusing post where someone use to believe this and then was amazed at the speed of the database versus the flat file, well put quite simple shove the flat file onto a ram disk and then do the test again.
Some databases use a form of compression, and whilst that can introduce an overhead it can also out perform non compressed at times, so yet again if that trick is being used then compress your flat file.
A database really is a suite of data access and storage functions, there is no more magic than that, and yes of course it becomes very complicated when looking for the generic and general way of storing and presenting all forms of data and processes. A DBMS has to work as an ecommerce site one moment, a pictur store the next, an airline booking system, an accounting system, a collection of searchable urls etc etc.
And therein lies the nub of this debate. If you are making a database or shall we use the term datastore, and that datastore is meant to be a huge datastore that is accessed by many people all at different times, and you are using many cheap nodes all connected to power this datastore, you would be a fool to install a standard DBMS system. Oh, the headaches you would give yourself, and your organization.
Conversely you would be foolish to install a similar datastore, to that described above, if you are building an application that will be accessed simultaneously by a hundred or so people that was not a standard DBMS.
The only thing I can think that is happening here is people think they should follow what Google is doing, and whilst there is a lot to be learned from Google, there is no point in going against the grain and using all the features they use, because frankly you will be using ideas just not suited to your project.
Google does not have to be accurate in their search results, their primary concern is to shift huge amounts of data about. And by accuracy I am referring to the ordering, they may have a tolerance of say 90% in that department, whereas in an accounting system you may be kinda miffed to order by price and have the results spewed all over the place, so accuracy required there is 100%.
Horses for courses really, the mapreduce technique is valid today when dealing with huge datastores, as performance increases and costs drop the traditional DBMS system will again become the better approach for that size of database, of course if datastorage requirements out pace the performance increases then mapreduce style ideas still have a role to play. I suspect that mapreduce will just be rolled into DBMS as yet another feature. This is the power of the MySQL database with the swappable engine model. But, what do I use, Postgresql I want to know where I am with the data.
Biting the hand that feeds IT
