Google open sources MapReduce compression Google has open sourced the compression library used across its backend infrastructure, including MapReduce, its distributed number-crunching platform, and BigTable, its distributed database. Available at Google Code under an Apache 2.0 license, the library is called Snappy, but Google says this is the same library that was …
..that there is something snappier, bigger, and more reductive under development in the Chocolate factory? Companies as big and subtle as Google dont just opensource key parts of their infrastructure like that.
An act of 'kindness' from Google means that there is some deeply evil malevolence afoot somewhere, probably involving crappy ads and privacy losses.
..mine's the one with the click thru banners in the pockets.
So basically Google have rediscovered what game developers were doing 2 decades ago? And network engineers probably 2 decades before that ;)
Back in the early 90's, faced with unpacking graphics from ROM to graphics RAM in real time, I used a variety of hand tuned, entropy coding free, codecs and achieved similar compression rates. It ain't rocket science, especially if it has to run on an original Gameboy or Sega's 8 bit consoles.
That's why they're giving it away, where Microsoft would BS the US PTO into issuing patents.
Still, having a standard scheme across vendors might be useful for net traffic... a bit like the compression modes modems used to use!
..that the difference is that Google have analysed the last digit of performance out of their algorithms, whereas you just got yours to work.
There really aren't that many computational processes that haven't been done before, and before and before. The difference usually comes in optimising the latest implementation to suit the environment in which it runs.
...you're assuming I (and the other programmers implementations I know of) didn't tune our compression codecs for speed and compression. Mine got continuously tweaked, measured and improved for >4 years. I didn't throw in 'similar compression rates' for no reason.
What is different is Google need to tune the compression end, not just decompression. I also did that but just because it was fun!
Faced with the same problem G chose the same old solution. Well known pattern recognition algorithms are blindingly fast, entropy coding never is without hardware assistance.
Apart from the question Why, is is good to see that some of the spirit of early InterNet days where people helped others out, still active in Sourceforge, etc., especially from a large entity like Google.
Likely it will help their competition, too, which is a sign of real philanthropy.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
