SwiftStack CPO: 'If you take a filesystem and bolt on an object API'... it's upside down

Thursday 7th January 2016 11:35 GMT Anonymous Coward

Snake oil

A disk drive is still fundamentally a block device: rotating disks divided into tracks and sectors.

The Kinetic drive might expose an object API, but there still must be a filesystem of sorts underneath, to map object names to blocks, allow objects to be created/modified/deleted, track free blocks etc. This idea of "upside down" is nonsense; you've just moved the filesystem layer into the drive itself.

In addition: how much trust do you have in this black-box filesystem buried in the drive firmware? Is it journalled? How good is it at handling fragmentation when many obejcts are created and deleted? What happens when it gets corrupted? How are the fsck and data recovery tools going to work, if you can't talk directly to the underlying blocks?

5 0 Reply
1. Thursday 7th January 2016 14:21 GMT SwordOfEnlightenment
  
  Re: Snake oil
  
  Swift doesn't care much about reliability of the underlying hardware / filesystems. One of the design criteria is that Swift "spreads" out the underlying files representing the objects and metadata over multiple drives / clusters by keeping multiple copies. Drive failure / file system failure / node failure is not relevant anymore, except of course for the poor operators managing the cluster ;-).
  
  2 0 Reply
2. Thursday 7th January 2016 14:28 GMT Simon Crowe
  
  Re: Snake oil
  
  I also think that the point with these drives is that the device does not operate in blocks anymore, i seem to remember seeing that for modern high capacity shingled drives maintaining a block mapping is expensive.
  
  So instead of having a block mapping table you have an object mapping table. The concept of blocks for a kinetic type drive does not exist
  
  1 0 Reply
  1. Thursday 7th January 2016 18:25 GMT Anonymous Coward
    
    Re: Snake oil
    
    It must have something akin to blocks - even if it treats a whole track as one large block.
    
    If a large object is stored, then it will have to span multiple tracks by being broken up into track-sized pieces.
    
    If lots of small objects are stored, then multiple objects will need to be stored on the same track (or else you waste a ton of space)
    
    Either way, you need a table which maps the name of an object to the track(s) or block(s) it is stored, and possibly the offset(s) within them.
    
    This can be just a simple non-hierarchical flat filesystem, like the early MS-DOS and Apple HFS, but a filesystem it is nonetheless. And it still has performance and reliability tradeoffs inherent to its design.
    
    However on a disk which exposes an object API, you have no choice but to use the filesystem they provide in the firmware.
    
    2 0 Reply
Thursday 7th January 2016 12:52 GMT Brewster's Angle Grinder

The wheel turns...

It strikes me that we've come full circle. The first filesystems weren't hierarchical. But having thousands of files in one directory would have been slow and unmanageable. Various competing concepts arose, but the hierarchical filesystem was the best and so, once Multics had invented it, it spread.

Now we can finally do a billions of "files" in a "directory". In fact that makes it easier, because the filesystem can easily span multiple physical devices. And so up pops object-store: and back we go to a flat directory.

2 0 Reply
1. Thursday 7th January 2016 14:31 GMT Simon Crowe
  
  Re: The wheel turns...
  
  The question is why was the directory concept introduced, i think it was for us humans as we couldn't comprehend thousands of names scrolling by us as we tried to navigate through the data.
  
  There is still a need for having a simplistic view of the data for persons to traverse and understand, but for large scale object stores accessed by your software directories are somewhat obsolete.
  
  0 0 Reply
  1. Thursday 7th January 2016 14:52 GMT Brewster's Angle Grinder
    
    Re: The wheel turns...
    
    But the directory was introduced for the computers; not for us. It's always possible to produce a view that shows what you want. ("Source code belonging to project X".) Servers on the web aren't really hierarchical and we manage.(And often the "filesystems" are virtual paths that are turned into a database query.)
    
    My argument was, I guess, that directories were a technical kludge that the technology is outgrowing. (They will no doubt return.)
    
    0 0 Reply
    1. Thursday 7th January 2016 15:40 GMT Anonymous Coward
      
      Re: The wheel turns...
      
      Why not both?
      
      One of the human-centric reasons for hierarchical folders was they give you name spaces, and in an OS such as Unix that treated every resource as being essentially file-like, that turns out to be really useful from a system-centric point of view. It allows you to have consistent naming distinguished by context.
      
      Unfortunately, ordinary non-technical people aren't very good at coping with hierarchical organization, it turns out; and somewhere along the way we lost sight of *why* we did hierarchies to begin with, and just kept doing them out of habit.
      
      0 0 Reply
      1. Thursday 7th January 2016 19:37 GMT Yaron Haviv
        
        Re: The wheel turns...
        
        People also used to have email sub-folders, and as the amount of emails piled up, just use the search bar
        
        Same with Object, you use the Metadata fields to find what you are looking for, and can have more context to work with
        
        Object storage doesn't need a full file system, can do with hash functions and has less dependencies between data structures
        
        Yaron
        
        0 0 Reply
Thursday 7th January 2016 20:03 GMT Triboolean

One thing directories do for humans is make it possible to partition by project, subproject, or other mapping so that you can have same name (such as settings.py, tracking.xls, project.mpp, readme) files without any worries about name collisions.

How long until you see names like toasters/designs/2015/jan/bread.file ?

Object storage is going to work well in some situations, not so great in others.

0 0 Reply
Monday 11th January 2016 18:08 GMT CaitlinBestler

Metadata is stored somewhere, preferably at one layer not two.

A file or object storage system store both metadata and data.

Having the storage server, or even a "smart" drive, store the metadata related to precise locations make sense. But it is hardly a new concept.

But a full local file system has a lot of naming metadata, which is handled by the object cluster. Writing that redundant metadata wastes CPU power, disk space and disk writes. It also creates more links that can be broken, with no way to use the redundant information for repair. It's a lose-lose proposition.

Using a pre-existing local file system may be a development trade-off worth considering, but it is far from something to boast about.

0 0 Reply
Friday 15th January 2016 16:23 GMT cloudguy

Kinetic represents change...

Well, if you listen to Mr. James Hughes whose team developed Kinetic, you will see that it eliminates storage servers and the need for a POSIX-compatible file system. The Kinetic key/value API plus Ethernet on the Kinetic HDDs is the innovation. The response from the object-based storage sofware vendors has been mixed. Mr. Joe Arnold from SwiftStack sees great promise in the Kinetic framework. Caringo has looked at it and said...meh, SWARM is better. Cloudian was of the opinion that it would need to implement a "split-brain" design in their software stack to use it. Scality was interested in seeing how it might be made to work with RING. Cleversafe (IBM) joined the Linux Foundation's Kinetic Open Storage Project, but not sure what they are actually doing with it. Not sure what Amplidata (WD) thought about it, but WD also belongs to the Kinetic Open Storage Project. HGST (WD) was also experimenting with their own Ethernet HDD that ran Debian, but this is not what Seagate is doing with Kinetic. Given that there has barely have two years of third-party development effort related to Kinetic, it seems premature to declare Kinetic a success or failure as an object-based storage architecture. Time will tell, even though Mr. Arnold is bullish on Kinetic.

0 0 Reply