Google goes back to the future with SQL F1 database

I wonder how many countries envy such a query and campaign power

I wonder how many countries envy such a query and campaign power. Why, imagine the things that could be done if one replaced "AdWords" with "Crimes and achievements" and "Campaigns" with "persons and events", and so on.

Eventually, every living person will end up in such a database, and all his or her birth, education, socioeconomic, social, criminal, work, and health factors will be on tap for anyone needing a quick socket wrench or disposable "instant agent", but one who does not know that s/he really IS a TOOL, so to speak.

OTOH, employment agencies (government or commercial) could instantly force unemployed people to take on jobs to continuously receive benefits until a real, sustained job arrives.

Anyone collecting fingerprints form dining room glasses; hairs and bodily fluids from hotel beds and hangers; blood in emergeency rooms, hair in barber shops and salons, ANYONE wanting to be paid for clandestinely forwarding such materials to some drop box could be an instant agent.

How soon before, as a matter of "business survival" before companies like Google, Yahoo, MS, and others fine-tune their meta data collection to mesh with "brokers"? Pretty soon, we all could be part of....

YuNiMaTrIx 001, attached as a node in some pri/sec/tert ajunct....(well, to bring the Borg into this), and we won't necessarily have Swedish family names...

And with Google behind F1 the 'cool kids' will either go back to SQL databases or will start rolling their own SQL-interpreters on top of their favourite NoSQL systems. I like the NoSQL approach for many use-cases but I have grown tired of inexperienced developers trying to use it for everything and proclaiming how great their achievements are when their problem never warranted a 'BigData' solution or a 'NoSQL' solution.

What amazes me is that Google is now in the position that IBM once held: whatever it does technologically, a large part of the IT industry will follow.

The point of what Google does is not SQL but ACID semantics in globally distributed systems, which is a landmark technical achievement. You should read this not as a step back but as a step forward for any storage systems that currently lack such semantics, including all conventional SQL engines, which only manage this on single node systems generally.

It doesn't quite solve that certain aspects of SQL don't scale that well (joins, distinct) and are probably a bad idea regardless of whether you can make them work or not. Though you do have a choice now. It doesn't quite solve the problem that there is a huge impedance mismatch between document and object oriented application designs and row/table oriented storage solutions. It doesn't quite solve the problem that most programmers need things like hibernate to shield them from what is a pretty rough and unforgiving language.

So, you say back to SQL, I say fully transactional, globally distributed document stores. I'll be likely to continue to prefer more expressive querying and indexing such as provided in e.g. Elastic Search; the power of doing proper declarative map reduce style processing such as provided in e.g. Cascalog, CouchDB, or Hadoop, the luxury of not having to break down my transactions into tons of sql calls, or the more powerful semantics of transactional graph databases such as neo4j. NoSQL is here to stay. It may just become properly consistent, eventually. The next ten years are going to be interesting.

As with many things in this game ...

... it eventually comes down to the right tool for the right job, and not using the trendy tech du jour because all the skinny-jean cool kids on the internet say you should. Sometimes the right thing is NoSQL, sometimes not.

As I understand it, Goolge Spanner is built using atomic clocks to synch data updates around the cluster, F1 is built on Google Spanner. This alone is going to make take up of the technology very restricted, unless Google offer access to F1 at a reasonable price.

Time to update the CV...

... with 5 years Google F1 experience

I truly think that much of the discussion around databases, and the scalability thereof, really misses the point.

Barely anyone really needs to scale. Most applications will never saturate a single, untuned mysql server, let alone anything with a bit more oomph.

The question I always encourage people to ask is, what is the data model that you need? Choose a database that implements the data model you need. Bending your app into the data model of a database you chose because it was 'scalable' is stupid.

Many applications don't fit the relational model, and many do.

If you design your application cleanly, and with the correct data model, it can be cleanly optimised to scale. If you try to ram your app into a shape that doesn't fit, it will be hard to optimise.

There is always the option of using more than one database, with different data models, at once, for different purposes.

Meh..

Release it as open source, then I'll pat you on the back and say well done!

Until then, this is just another "look at our shiny toy" lol

Google, NuoDB, and the rest

I concur with the fundamental thesis of this article that it will be hard for anyone to replace SQL for most database use cases / applications. NoSQL reminds me of the object-oriented database days, when the claim was made that it will replace relational database technology.

What is true is that relational database technology must evolve to support scale-out datacenter architectures. The likes of Google and NuoDB are well on their way to deliver on that promise. For NoSQL to grow beyond a 100M dollar business, it will have to embrace SQL in some form or the other.

Or just use PICK

Rather than going back to SQL after dumping NoSQL there's always the PreSQL databases that use the PICK model.

I'm sure google only employ the finest, brightest and most optimal specimens of humanity, but for the love of God, they come across as such fucking pricks, they really do.

Transactional consistency is not just good for databases, but filesystems too!

NoSQL has always seemed like an f'ing dumb idea to me, and MySQL is a dated toy; proper SQL databases rock, because tuned SQL and query optimisers can do joins and aggregation a lot better than coders, so light mapping frameworks like MyBatis, thrash bloated NIH ORMs like Hibernate.

Use a proper free database like PostgreSQL, look as scalability info. sites and learn; just shard the data already, or riff on these ideas from Google.

Maybe this will also kick Operating System providers to see that logging (asynchronous) filesystems are junk which should have been phased out years ago, and replaced by a concurrent, queued, transactional model filesystem like ZFS; then I wouldn't see intermittent, race condition faults (NTFS in Windows is pathetic), and the painfully costly downtime to run off-line repairs on filesystems! Kudos to PCBSD and FreeNAS dists. for getting this right with ZFS. Much like a proper RDBMS, ZFS also has advanced features for free e.g. enforced data integrity, integrated RAID, datasets, snapshots, encryption, and de-duping.

It annoys me that some coders have't made the effort to property grasp concurrent development, because it can be quite time consuming to have to retroactively fix code where people either didn't think about concurrency, lazily used blocking locks, or naively left gaping holes for race conditions! Most code is or will become concurrent, even when you naively think not.

What I found ridiculous is that when explaining why they do not deem Spanner relational, Google said it does not accept tables without keys. Now, what violates the relational model in SQL is precisely allowing bags, that is, tables without keys, which are not relations. It that is the point, it is precisely what makes Spanner relational!