Sign in / up

back to article Google's MapReduce suddenly not so backward

What was seen as a major hole in Google's MapReduce database technology has been plugged, not once but twice. In the same week. Californian start-up Aster Data and its more established rival Greenplum have both launched SQL integration for MapReduce. The lack of SQL tools was one of the main criticisms levelled at MapReduce …

COMMENTS

House rules Send corrections
This topic is closed for new posts.
  1. James Anderson
    Happy

    Ahem

    Anyone looked at ORACLE cluster or DB2 Sysplex recently.

    They obviously use some variation of MapReduce internally to spread the laod over several machines, its just not exposed in hte API.

    And as I said at the time Google's BigTable does support an SQL varient, among several options for accessing the database.

    0 0
  2. Anonymous Coward
    Anonymous Coward

    Great, now miss out the SQL

    That's great, now you can make it faster by not putting the data in the SQL database in the first place...saving the query optimization to get it out again and shoving it into your mapreduce directly.

    As Jeremy recently discovered:

    http://jeremy.zawodny.com/blog/archives/010523.html

    Remember folks, SQL databases are for data that can be

    1. Run against a query that can be expressed in SQL syntax

    2. That query can be improved by one or more indexes

    3. The burden of making the index is less than the time it saves in your queries.

    If it can't be expressed in SQL then the query can't be reduced and you end up reducing the data instead..... which is the point of map-reduce.

    Test: Given a changing set of points in 3d, run a query against a snapshot of that data that returns the nearest point to (x,y,x)... feel free to express that in sql....

    0 0
  3. Alistair Wall

    @AC

    select min(id) keep (dense rank first order by ((x-p_x)*(x-p_x) + (y-p_y)*(y-p_y) + (z-p_z)*(z-p_z)) ) from points

    (Oracle syntax)

    0 0
  4. Torben Mogensen

    Sawzall

    Google has made their own query language, called Sawzall, on top of MapReduce. It doesn't look much like SQL (it looks more like a "normal" programming language) , but it seems quite nice.

    0 0
  5. Anonymous Coward
    Anonymous Coward

    @Alistair

    "select min(id) keep (dense rank first order by ((x-p_x)*(x-p_x) + (y-p_y)*(y-p_y) + (z-p_z)*(z-p_z)) ) from points"

    i.e. brute force, (x,y,z) is what you query this db against, so keeping the index doesn't help you because x,y,z changes and would be recalculated each time.

    Map reduce. No point in putting it in the db either, just bung the raw data into a map reduce.

    0 0
This topic is closed for new posts.

Other stories you might like