Nowadays, when it comes to crunching epic amounts of web data, Google's MapReduce credo is all the rage. The Mountain View method of distributing back-end compute tasks across a sea of commodity machines has given rise to the open source Hadoop platform, which now underpins Yahoo!, Facebook, and even a chunk of Microsoft Bing. …
Back to the Future...
The idea that MapReduce might not be the be-all and end-all of parallel computing shouldn't be surprising. Consider the more "traditional" field of large-scale scientific computing (i.e. solving PDEs). MapReduce is absolutely NOT the fundamental way to get parallelism because the different parallel parts need to communicate on a fine-grained scale.
The surprise to me was that something as simple as MapReduce was able to solve AS MANY problems as it has.
I'm not sure?
So does this mean Yahoo! considers my web connection a distributed resource? Maybe I should send them I bill for using my leccy and PC? Hey - what about the cost of lighting, heating, beer, CPU cooling,coffees,biscuits? OMG - spreadsheet required...
Re: Back to the Future...
That shouldn't really be surprising. As they say, when all you have is a hammer, everything looks like a nail.
Ever since MapReduce was described in that seminal white-paper, it became all the rage, and a lot of people did see it as the end-all, be all or distributed processing; and I posit that *because* of that, a lot of people have replaced their entire toolbox with this single hammer.
So then, it's not that MapReduce is able to solve so many problems, it's that it's been applied to so many problems, irrespective of its disposition to solve such problems.
- Hi-torque tank engines: EXTREME car hacking with The Register
- Product round-up Ten excellent FREE PC apps to brighten your Windows
- Review What's MISSING on Amazon Fire Phone... and why it WON'T set the world alight
- Product round-up Trousers down for six of the best affordable Androids
- Why did it take antivirus giants YEARS to drill into super-scary Regin? Symantec responds...