1 post • joined 12 Aug 2009
GFS2... truely awe-inspiring
We (the guys in charge of moving all the data from GFS to its successor, and the day-to-day maintenance) had a good laugh in the office about the comment "wearing massively reinforced underwear". Sometimes it's better not to wear underwear when doing these sorts of upgrades...
As for "other tools"; Lustre was invented as a local network filesystem. GFS was invented to handle thousands of tasks all reading & writing as fast as they could all day every day. The indexing pipeline; download the internet, index it, run a few mapreduces over it to mark down spammy sites, crappy sites, duplicate sites, dead sites etc. and then compress it so it could be shipped all over the place. As Sean says in his interview, these days 'routine use' is dozens of petabytes of data that has to be randomly accessed - as in, the metadata has to stay in RAM.
- Analysis iPhone 6: The final straw for Android makers eaten alive by the data parasite?
- Stephen Pie iPhone 6: Most exquisite MOBILE? No. It is the Most Exquisite THING. EVER
- First Crack Bloke buys iPHONE 6 and DROPS IT to SMASH on PURPOSE
- Early result from Scots indyref vote? NAW, Jimmy - it's a SCAM
- First Fondle Register journo battles Sydney iPHONE queue, FONDLES BIG 'UN