back to article Peeking up the skirt of Microsoft's hardy ReFS

As reported this past fortnight, Microsoft's new Storage Spaces for Windows 8 is only half the story; the operating system builder is also throwing in a new Resilient File System (ReFS) while retaining most NTFS features and semantics. Storage Spaces is a Windows 8 feature that enables a PC user to aggregate physical disk …

COMMENTS

This topic is closed for new posts.
FAIL

More lock-in

I guess FUSE-NTFS was getting too good, so now they need to migrate to another proprietary filesystem in order to continue their lock-in scheme...

7
5

I think you're being a bit melodramatic about this. The only filesystem that has those kind of features is ZFS as far as I'm aware and it probably doesn't support the kind of things Windows requires (and is under a license which MS probably don't want to sign up to). I doubt their prime concern here was killing open source compatibility, although I suspect it came up. I also don't see anything here which precludes you continuing to use vanilla NTFS.

3
1
Boffin

actually,

CDDL code would be vary easy for them to incorporate into windows. The CDDL is only scoped to the actually covered files (unlike the GPL is it non-viral). Therefore they would not have to worry about making any changes to how windows is licensed, they would just have to provide the code to their version of the specific CDDL licensed files they used under the terms of the CDDL, not the entire OS (nor even the other original files created as part of the porting process).

I'm curious, what things do you think windows needs that ZFS does not do?

3
0

So...

...how does it stack up against the likes of BTRFS (also B-tree) or Ext4 or any of the other options available?

This just smacks of "N.I.H." syndrome and, considering it's MS, "Using an open system would increase competition".

7
0
Ru
Headmaster

More 'live by the sword...' than NIH

You can't go around threatening all and sundry with software patent infringement unless you're absolutely certain that your own stuff is unimpeachable. MS have come unstuck here in the past as I'm sure you're aware, but ultimately the safest way to write software in the sort of Mutually Assured Destruction business environment in which MS likes to operate is to write everything yourself from the ground up, using designs you've developed yourself.

They've basically painted themselves into a corner, and that's why they product so much "me, too!" stuff instead of using liberally licensed open source software.

9
4
Thumb Down

Other OS support?

So, how much information will MS provide to help us create Linux drivers for this file system? Or will we have to reverse-engineer it from scratch? I kind of suspect they will make it as difficult as possible, even to the extent of claiming copyright and/or patent privileges...

6
0
Anonymous Coward

As long as I can delete my data and be confident I have actually deleted it I will be happy.

Otherwise I suspect a flag will be set in metaddata showing the file as deleted so it cant be seen.

1
0

another file system

why another file system ?

whats wrong with the other ones out there

apart from they are not Microsoft locked.

how can companies and governments insist on open file structures, but not open file systems ?

if I plug a disc into a different system I'd expect , with the appropriate pass words etc, to be able to read it,

Ha

3
5
Linux

Because Micros~1 doesn't want to slip more than ten years behind Linux?

The kind of features they are including with their "new" file systems put Windoze on a par with Linux in the mid-noughties as opposed to NTFS's equivalence to the mid-nineties.

Aggregated storage? Jeez! Amazing! I forget how long ago I first toyed with LVM on Linux!

So we may have to use our Windoze VM for a while to retrieve data from borked ReFS storage while waiting for stable kernel drivers. No biggie.

5
6
Anonymous Coward

Aggregated storage is not what you think it is

Windows can do aggregated storage in the Linux sense: that is, mount a filesystem somewhere in an existing filesystem. But this isn't terribly helpful if you need extra space somewhere else in the filesystem hierarchy.

What people actually want to do is have a single filesystem transparently spanning multiple drives, preferably with the ability to add and remove drives without rebuilding the filesystem. ZFS can do this now. Btrfs implements this on Linux, but it is in no sense production ready. ReFS will add this for Windows.

3
3
Flame

LVM is not what *YOU* think it is - it *IS* aggregated storage

Try to engage your brain before you call someone else stupid. Yes we *ALL* know that mounting disk partitions in a filesystem is not aggregated storage.

Look up LVM - Logical Volume Manager. If you don't know how to use Google here's a link: http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/

LVM *IS* aggregated storage, it *IS* used in enterprise storage solutions, it has been around in Linux since the late nineties and has been a mature product for 5+ years.

5
3
Vic
Silver badge

> What people actually want to do is have a single filesystem transparently

> spanning multiple drives

Like LVM, then.

Vic.

6
1
Vic
Silver badge

> LVM *IS* aggregated storage

... *can be* aggregated storage. can be many things, in fact.

It's quite wonderful to watch people's faces the first time they see an lvresize / resize2fs play...

Vic.

4
0

Key features are NOT in common products

Actually, as nice as the easy management features sound, the real strengths here are features such as checksums for detecting silent data errors, copy-on-write, etc.

-d

0
0
Anonymous Coward

@Ian McNee

Tee hee - you're twelve years old, you? All-capital words and asterisks too! The Reg should make a stamping foot icon for you ...

0
4
Silver badge

The pertinent point

Is this is for Windows Server edition. It won't be in bog standard Windows Paeon edition. It also doesn't support booting from ReFS. Nor can you convert a disk from NTFS to ReFS despite them sharing some similarity.

This to me suggests Microsoft is being cautious. Server edition gets it first, but it's unlikely to show up anywhere else until Windows 9 or some major service pack.

5
0

It's also interesting in terms of SKUs

The blog explicitly references "Windows 8 Server" which automatically deprecates the "Server 20xx" naming convention (for the better, I think) and leads one to wonder whether MS are attempting a simultaneous Server/Desktop release.

They haven't done that since Win2K.

2
1
Anonymous Coward

@dogged

Windows 7 and Windows Server 2008 R2 simultaneous client & server SKU releases, built from the same source branch (not just staggered by 1 service pack like Longhorn).

0
0
Thumb Up

Checksumming!

Microsoft have (finally) seen the advantages of ZFS, and implemented (a few) of the the features. There seems to be a fair bit of conflicting info floating around on the web, but if ReFS with Storage Spaces does have full checksumming then I might consider it as an upgrade path for my Solaris / ZFS file server.

Yes, I agree, it'd be nice if they'd play nicely with open source and help get BTRFS up to production standard (or somehow make a deal with Oracle and use ZFS) but failing that this is a good second best.

2
0
Anonymous Coward

Err...

To be fair, ZFS was hardly in wide scale use in 2008 when the last NT server was released.

0
0
Unhappy

There are many NTFS features that will not be present on ReFS

There are many people that are not happy about the loss of named streams, hard-links, and compression.

From the Q&A at the end of the 'Building Windows 8' blog entry:

Q) What semantics or features of NTFS are no longer supported on ReFS?

The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas.

5
0
Silver badge

Hang on,

doesn't Windows itself depend on some of those features? Hard links for the WinSXS folder, or am I mistaken?

1
0
Anonymous Coward

Win SXS

It actually uses reparse points which are links but for linking directories. From Vista onwards there are file and directory links, it appears that it's the file links that are being removed. It's a shame that they're dropping this as POSIX is a nice to have, I quite like having SFU installed on a server and this would seem to suggest that the new filesystem can't be used with SFU.

If you're interested a common WinSXS reparse point you could look at is:

fsutil reparsepoint query c:\windows\assembly\gac_32\system.enterpriseservices\2.0.0.0__b03f5f7f11d50a3a

1
0
Childcatcher

We use compression extensively on our Windows servers. What is going to happen to us? They just don't care...

0
0
Anonymous Coward

@Mark jacobs

They're not making you use ReFS, NTFS is still supported, so it looks like they do in fact care.

0
0

WinFS

Whatever happened to the file system as database?

1
0

It's here

ZFS and BTRFS to name but two (with varying degrees of implementation).

Of course, Windows can't use them.

0
0
Anonymous Coward

So does this mean I won't have to defrag my file system any more?

Seriously; what kind of shitty server OS requires me to do that *cough* Windows Server 2008 *cough*? Unbelievable.

2
4
Facepalm

Ugg

Please M$ do us all a favor and stop with the half assed, brain dead copies of other peoples file systems.

You know it won't work, not at least until version 3 with version 5 being the one that everyone expected in version 1.

All you have to do is integrate ZFS and even the license still lets you stay kind of closed source.

Copy Apple if you must.

4
8
Anonymous Coward

Seriously?

Is that the best comment you've got?

MS implement functionality that's available elsewhere and it's brain-dead copying, let me guess, when Linux vendors do it (let's take journaling file systems as an example) it'd be implementing best practice?

2
4
Meh

Allocate-on-write

Allocate-on-write and (although it doesn't say so) ordered metadata update? Are they going to (try to) patent it?

George 3 1965.

2
1
Silver badge

File systems are proper hard

Wonder how long its been in development? The blog says they've given it loads of thrashing with their test cases but that is a far cry from the real world.

No matter how hard you play in the lab, users will find another "corner case" that you missed.

My favourite example of this is an image (say from dd) of a ReiserFS3 lies on a RFS. One day you run fsck (chkdsk for the uninitiated) and find the image and the FS intertwined because the checker had a hard time telling the difference between them. Don't know if this feature is confined to RFS either.

Read the online discussions from FS developers and you'll wonder why you use anything that isn't at least five years old in the real world ...

Finally, no matter how fancy it is, you'd better back it up properly and give it a UPS and a good hardware RAID or SAN if you care about your data.

Cheers

Jon

2
0

@gerdesj

"...Finally, no matter how fancy it is, you'd better back it up properly and give it a UPS and a good hardware RAID or SAN if you care about your data...."

Backups are good, but how do you know that the backup does not contain corrupted data? I have heard of old backupped files, and long time after, the files are not readable. The zip file is not extractable, the media mp3 is not playable, etc.

A backup needs checksums to detect and protect against these data corruption errors. That is the reason MS is adding checksums for metadata. However, that does not suffice, the data need also to be checksummed. Thus, ReFS is not that safe. As research shows, NTFS is not safe either. Nor is XFS, ReiserFS, JFS, etc which researchers prove. There is a reason ECC RAM uses checksums, so should filesystems.

Thus, a backup on a hardware raid or SAN does not suffice, and you might still have corrupted data. There are many texts on this, I hope you have not missed them.

1
1
Devil

NTFS certainly does lose data.

Hope the new error-detection highlights the cases where NTFS loses the entire tail-end of a directory (and all the respective files have disappeared).

2
1

Rather confused by the fact that this will come out for the server version of the OS, but is missing features (that they have "chosen" not to include) such as file level encryption and quotas which are exactly the sort of things it would seem to me that you're likely to want in a server rather than a desktop. Maybe they're opening up the market for a bunch of third party products to provide the missing bits?

1
0
Anonymous Coward

Err

Surely you want encryption on the desktop side, servers should be nicely secured both physically and logically, anyone can get their nite on a laptop/desktop.

0
0
Silver badge
Linux

I wonder how many patents MS has on this lot

They certainly won't want those pesky freetards reverse engineering it now will they?

4
0

Well it is certainly greater than the number of valid patents they hold.

2
0
Linux

Surely the EU-mandated

interoperability requirement means that MS have to allow interaction with ReFS, & thereby reveal enough information for open-source drivers to be written? Well, let's hope it does anyway.....

0
0
Anonymous Coward

Err?

Interoperability would be the ability to read and write files with no extra hooks into the filesystem available to MS than are available to everyone else. It would not be assisting rival companies to develop your technology for their competing platform.

1
0
Silver badge

SMART Virtual Machine Self Actualisation ..... Remote EnigmatICQ Virtualisation

" and data is verified and auto-corrected using checksums" …

Ah, yes, those Magic Semantic Algorithms in Service of Universal Singularity …… which suggest that do indeed Exist and are in Present Future Virtual Play, Microsoft/Surendra Verma/Azure.

A lady on the radio has just said that she is passionate and not hysterical, and indeed she was and talked perfect sense.

0
0
Mushroom

Hmm

Before I make my comment, I would like to point out that by no means am I a Microsoft fanboy (I have a server running Citrix (Linux) splitting it up into virtual servers running a mix of Windows or Linux depending on the workload being served) or trying to defend Microsoft in any way but...

If Microsoft said "Ok, so this new FS we are after needs to basically cover all the functions currently offered by the Linux LVM system, so for the sake of ease of implementation and compatibility lets just implement LVM" then the Open Source zealots would haul Microsoft up in front of the first court that would take their case demanding that the Windows code base be opened up to everyone.

I understand the need for open and available specs and drivers for filesystems, but for Microsoft to even think about risking implementing an Open Source filesystem in Windows then they would be extremely negligent to their shareholders and just bad business sense. The only logical and legal choice they have is to implement a new custom closed source filesystem.

While it doesn't do a lot of things NTFS can do and does a lot of things NTFS can't, it still sounds like a pretty decent filesystem, While the whole FAT/FAT32 thing was a bit of a debacle, being an old legacy filesystem, NTFS was actually pretty good, so I am looking forward to giving ReFS a try to see how it performs. I also think the EXT filesystem is pretty good and I would love a spare box to give ZFS a try!

*Nuke: Closest thing referencing the bunker i'll need to deal with all the inevitable flaming I am about to get!

1
2
Vic
Silver badge

> the Open Source zealots would haul Microsoft up in front of the first

> court that would take their case

This is untrue.

If Microsoft were to implement LVM, the entire Open Source community would say "excellent - well done".

If Microsoft were, instead, just to take the LVM source code and incorporate it into Windows, the copyright owners (primarily Red Hat) would insist on the source being open according the the GPLv2 licence under which it is distributed.

This is the principal difference between open- and closed-source; open-sourcers *want* stuff to proliferate. The GPL, for example, gives you explicit rights to study the code with no restriction on the purpose of that study; you are *encouraged* to find out how it works and then write your own version.

All that is really frowned upon is blatant copyright infringement. MS got caught out that way once before, and were obliged to release some of their code under GPL. I can't see them doing that again for many years...

Vic.

2
0
Thumb Up

Client side too please.

Now just put this in Windows 8 Professional and make 85% of El Reg readers very happy. The other 15% can continue to whine here and not benefit from it.

0
3
This topic is closed for new posts.

Forums

Biting the hand that feeds IT © 1998–2017