File system trees are inefficient and slow when locating files in a filespace occupied by billions of files and folders. Storing the data as objects in a flat storage space is becoming a recommended alternative. But, as soon as you go for object storage to defeat this file system tree traverse problem, you face a fresh problem: …
Testing 10,000 nodes
They could run 10 VM instances per physical node. Of course that'll introduce inefficiency, but it should be just a constant VM overhead, and will let them see if anything breaks down or scales badly.
Not exactly new
This same sort of DHT or consistent-hashing approach is already widely used. Amazon's Dynamo and Berkeley's OceanStore both work essentially this way for different kinds of data, and have been around for over ten years. Atmos, which you even mention, inherits some of the same ideas from OceanStore. Tahoe-LAFS and GlusterFS both embody those ideas for files, Voldemort/Cassandra/Riak for key/value stores, etc. Scality might indeed have some very good technology, but the DHT part isn't their differentiating feature.
MPAA to send take down notice to all these Distributed hash table P2p for profit any day now ;)
MPAA and others , perhaps Antipiratbyrån (Anti-Piracy Bureau, APB) in the EU OC, to send take down notice's to all these Distributed hash table P2p for commercial profit companies any day now ;)
after all Distributed hash table and P2P are illegal the world over or so they would have you believe so Distributed hash table P2P coming to the cloud can only mean Piracy right, or that will be how they spin it if it suits their everything's piracy because we say so agenda