back to article Ghost of DEC Alpha is why Windows is rubbish at file compression

Microsoft's made an interesting confession: Windows file compression is rubbish because the operating system once supported Digital Equipment Corporation's (DEC's) Alpha CPU. Alpha was a 64-bit RISC architecture that DEC developed as the successor to its VAX platform. DEC spent much of the late 1990s touting Alpha as a step …

Anonymous Coward

Re: So why not create a new v2 compression scheme?

How often are you going to remove an NTFS drive from a Windows machine and try to install it in an older Windows machine?

Isn't Windows riddled with DRM 'tilt bits' to stop this sort of 'own your own data' nonsense from ever happening?

1
4
Anonymous Coward

Re: So why not create a new v2 compression scheme?

"Isn't Windows riddled with DRM 'tilt bits' to stop this sort of 'own your own data' nonsense from ever happening?"

Something along those lines. It's what MS mean by "Trustworthy Computing": high value content must be protected everywhere on the path between the content rights owner and the viewer/listener.

0
0
LDS
Silver badge

For an interactive system "wait" means poor user experience.

Yes, when the main thread blocks waiting for I/O to complete. That's why async I/O, queues, completion ports, and the like are welcome - you say the OS "read/write this data, and notify me when you did, meanwhile I can still interact with the user, for a good experience"

0
0
Mushroom

Hmmm

Windows 2.11 anyone? It seems that's where entropy Microsoft style is going to take us.

0
3
HCV

One Step Beyond?

"...touting Alpha as a step beyond anything else on the market at the time. It was mostly right: x86-64 didn't arrive until the year 2000."

Er, what? Even if you define "the market" as being "the market for processors desperately hoping to split the 'Wintel' atom", that still isn't true.

3
0

Re: One Step Beyond?

The first date in the Wikipedia article is 2000, let's go with that.

Oh, the Opteron didn't come out until 2003? Eh, shrug.

0
0
Silver badge

Re: One Step Beyond?

One Step Beyond!

2
0
Silver badge
Linux

Yet another thing Microsoft sucks at

Even the humble Amiga had transparent (using magic-number style datatype handlers) and system-agnostic (multiple Amiga OS and CPU versions) compression in the form of the XPK, XFD and XAD libraries, with a plugin architecture that supported multiple compression and even encryption algorithms (including arch specific variants).

Speaking of the Amiga, did you know that Microsoft nicked their CAB compression tech from LZX, a once popular replacement for the LHA format so ubiquitous in the Amiga world?

This is the main problem with Microsoft. In the words of Arno Edelmann, Microsoft's then European business security product manager; "Usually Microsoft doesn't develop products, we buy products".

What Edelmann failed to mention, but which we can easily deduce from decades of experience, is that Microsoft consequently has no idea what to do with these assimilated products, and just sticks them together with duct tape, spray paints a Microsoft logo on them, then crosses its fingers.

To really grasp why Microsoft is so technically inept, you need to understand that it isn't actually a software development company, it's just a sales company, and salesmen make lousy software engineers.

27
8
IT Angle

Re: Yet another thing Microsoft sucks at

To really grasp why Microsoft is so technically inept, you need to understand that it isn't actually a software development company, it's just a sales company, and salesmen make lousy software engineers.

Or, as we used to say 'Designed to sell, not to work'

20
1
Silver badge

Re: it's just a sales company

As opposed to all the companies that don't sell anything like.....?

0
2
Silver badge

Re: Yet another thing Microsoft sucks at

Microsoft also had code supporting specific features of the Amiga hardware in rather earlier versions of their OS. Annoyingly I couldn't find it the last time I looked for it (I found it by accident originally) but it was definitely there somewhere buried in the depths of the graphics handling code.

0
0
Silver badge

Re: Yet another thing Microsoft sucks at

As I recall, it had to do with the differences between Amiga's bit-plane orientation vs. the god-forsaken way the IBM-derived way of handling graphics. Theoretically, I still have a copy in storage, if they haven't bit-rotted away. I stumbled across all sorts of tidbits concerning other platforms algorithms hidden in the MSDN discs. Probably not intentionally hidden, just required try and try again in their sucky search.

0
0
FAIL

"Chen says one of his “now-retired colleagues worked on real-time compression, and he told me that the Alpha AXP processor was very weak on bit-twiddling instructions".

I guess that Chen doesn't know what the "R" is "RISC" means!

mb

4
13
Gold badge

Really? I'd say that Chen has a pretty good idea of what it means and explains it clearly in the blog. In this case, it means that the processor was very weak on bit-twiddling instructions.

"Reduced" implies that you've taken a broad view of what's important and prioritised that stuff, so there will inevitably be niches that you aren't serving with dedicated instructions. Here, apparently, is the niche that the Alpha's designers chose not to serve. There's no blame or moral opprobrium attached to it -- it was simply an engineering trade-off.

19
1
Silver badge

byte, word and longword addressing

The earlier 'classic' Alpha processors (before EV56) did not support byte or word boundary aligned reads and writes from main memory. In order to read just a byte, it was necessary to read the entire long-word (32 bits), and then mask and shift the relevant bits from the long-word to get the individual byte. This can make the equivalent of a single load instruction from other architectures a sequence of a load, followed by a logical AND, followed by a shift operation, with some additional crap to determine the mask and the number of bits to shift.

But you have to remember that in the space of a single instruction on an x86 processor, an Alpha could probably be performing 4-6 instructions (just a guess, but most Alpha instructions executed in 1 or 2 clock cycles compared to 4 or more on x86, and they were clocked significantly faster than the Intel processors of the time - RISC vs. CISC).

Writing individual bytes was somewhat more complicated!

I was told that this also seriously hampered the way that X11 had to be ported, because many of the algorithms to manipulate pixmaps relied on reading and writing individual bytes on low colour depth pixmaps.

5
0
Anonymous Coward

Re: byte, word and longword addressing

Byte handling was much improved in the later Alpha processors - another case of a big company getting it wrong first time around.

4
0

I'm not sure that calling it rubbish is even all that accurate -- it's not bad for what it is, and competing modern options like LZO and LZ4 aren't much better, they're mostly just faster. It's annoying that they didn't include both a fast and slow compression, like they did with cabinets and wim, but I understand that they solved the 90% problem and going beyond that would just mean new UI work and lots more testing.

What's rubbish is that fact that it's not used by default on all installations since 2004 or so, by which point the disparity between CPU overhead and reading from disk had become completely absurd and file compression was rock-solid. Every OS since XP SP2 should have made it mandatory; it basically halves the overhead of OS and program installs, and is like a little extra space for everything else.

1
1

Actually, most of the data stored on the disc is probably already compressed (multimedia, etc.) and OS would simply waste CPU cycles with another layer of, in that case, pointless - or even counterproductive (as in: more data needed to store what OS sees as random sequence of bytes) compression.

File-system level compression might had its day when a) the prices of storage were much higher and b) where most files stored on the medium were compressible. Those were the days of DoubleSpace / Stacker and similar products from the 1990s.

Now, it could be still useful for compressing the OS / application executable binaries but the gains to be got from that are simply diminishing (say, you get 30% compression - how much does that help as opposed to increased CPU loads / latency due to on-the-fly compression/decompression)?

Having file system compression as a feature can definitely be useful but having it ON by default? Definitely not. For most typical usage patterns where most of the I/O would be actual application data, not binaries, it would just waste CPU cycles trying to compress something already compressed.

7
0

Back in 2003, when I first made the attempt to offline compress the OS, it was an absolute night-and-day performance difference in startup and daily use, thanks to how crappy hard drives of the day were. I didn't say on the full disk, I just said to use it by default; Microsoft could have improved almost everyone's experience for little effort, even if it was only for the Windows and Program Files folders.

Now I have an SSD, and only enable compression on disk images and the OS to fit a little more until I can upgrade it. Performance difference is pretty much zero, when I've benchmarked, because the overhead of compression was designed to be low for 20-year-old CPUs -- it's undetectable now. (Unless you force LZX mode, which I'm too lazy to.) Sure, the SSD itself would compress for performance purposes, but it won't actually give you back any of that extra space.

For the external mass-storage disk, of course, there's little point in bothering.

The days of resource constraints that can be relieved by workarounds aren't behind us for everyone just yet.

0
0
Silver badge
Headmaster

Re: "chose not to serve"

The whole point of CISC was to save memory. It was never anything but a kludge, and these days a wholly unnecessary one. In retrospect it also seems like a crutch for lazy programmers. Mostly it's just a circular-dependent legacy we're stuck with, like the Windows ecosystem.

3
11
Silver badge

Re: "chose not to serve" @Oh Homer

CISC processors predated the adoption of the terms CISC and RISC. While you could say that, for example, a 6502 microprocessor was an early RISC processor, it was not really the case. The first processor that was really called a RISC processor was probably the Berkley RISC project (or maybe the Stanford MIPS project), which pretty much branded all previous processors as CISC, a term invented to allow differentiation.

As a result, you can't really claim any sort of design ethos for a CISC processor. Saving memory was a factor, but I don't really think that it was important, otherwise they would not have included 4 bit aligned BCD arithmetic instructions, because these wasted 3/8ths of the storage when storing decimal numbers.

You can say the converse. RISC processors, especially 64 bit processors often sacrificed memory efficiency to allow them to be clocked faster.

5
0
Bronze badge

Re: "chose not to serve"

The whole point of CISC was to save memory. No, No, a thousand times NO

The whole point of CISC is to save memory bandwidth

The first real CISC was the PDP11 - and you can see it clearly in the instruction set - the object is to fetch as few bytes of instruction from memory and then do as much as possible before you have to do it again! It doesn't matter a toss how fast your instruction cycles are if your CPU is sitting there waiting for the loads and stores to complete so it can fetch another instruction.

Sure you can make the problem harder to identify with gloriously complicated caching and pipelining, but the physics remains the same.

CISC was the winner because main memory is DRAM (slow as hell) while the CPU's internals are static RAM - massively faster - and the communications between them are a serious bottleneck. L1 and L2 cache help - a bit - but "ye cannae break the laws of physics, Captain".

6
0
LDS
Silver badge

"they would not have included 4 bit aligned BCD arithmetic instructions"

IIRC "packed" BCD instructions had performance tradefoffs and limitations compared to the unpacked ones - i.e. multiplication and divisionis supported only for the latter type.

1
0
Silver badge

Re: "chose not to serve" @Loud Speaker

That's a very interesting point, one I had not thought about, but the term CISC actually refers to a Complex Instruction Set Computer, and is defined by the number of instructions in the set, and the number of addressing modes that the instructions can use. I would say that the memory bandwidth savings were secondary, especially as most early computers processor and memory were synchronous.

I'm not sure that I totally agree with the definition of a PDP11 as a CISC (although it was certainly several generations before RISC was adopted), but the instruction set was quite small, and the number of addressing modes was not massive and exceptionally orthogonal, so it does not really fit in to the large instruction set many addressing modes definition of a CISC processor.

What made the PDP11 instruction set so small was the fact that the specialist instructions for accessing such things as the stack pointer and the program counter were actually just instances of the general register instructions, so were really just aliases for the other instructions (you actually did not get to appreciate this unless you started to look at the generated machine code). In addition, a number of the instructions only used 8 bits of the 16 bit word, which allowed the other 8 bits to be used as a byte offset to some of the indexing instructions (contributing to your point about reducing memory bandwidth).

One other feature that was often quoted, but was not true of most early RISC processors was that they execute a majority of their instructions in a single clock cycle. This is/was not actually part of the definition (unless you were from IBM who tried to redefine RISC as Reduced Instruction-cycle Set Computer or some similar nonsense), although it was an aspiration for the early RISC chip designers. Of course, now they are super-scalar, and overlap instructions in a single clock cycle and execution unit, that is irrelevant.

Nowadays, it's ironic that IBM POWER, one of the few remaining RISC processors on the market actually has a HUGE instruction set, and more addressing modes than you can shake a stick at, and also that the Intel "CISC" processors have RISC cores that are heavily microcoded!

2
0
Silver badge

Re: "chose not to serve" @Oh Homer

I was referring to its evolution rather than its inception, an evolution that mainly Intel took to perverse extremes.

0
0
Anonymous Coward

Alpha wasis a 64-bit RISC architecture

You'd be amazed at how many DEC Alpha machines still run critical elements of the UK's infrastructure.

Anonymous as I probably shouldn't say that out loud...

7
1
Silver badge

Diversity not completely dead

6 Alphas show up on Debian Popcon. That puts Alpha head of M68k and S390, but behind prehistoric versions of ARM and MIPS.

0
0

Re: Diversity not completely dead

Seeing as Debian dropped support for Alpha, are you really that surprised?

I image most are running Tru64, OpenVMS, or NetBSD.

0
0
Windows

Obvious bull, redux

I always imaging Microsoft's development approach to be akin to a herd of cows kept in the same field for years who all learn to shit under the same tree.

4
1
Anonymous Coward

Horshit!

"I always imaging Microsoft's development approach to be akin to a herd of cows kept in the same field for years who all learn to shit under the same tree."

Correction from a rural correspondent:

Horses will pick a spot in a field to shit in and use that. Cows don't care where they shit.

2
0
Anonymous Coward

Did they track the various revisions to the Alpha instruction set as newer processors were released, or was the whole thing a just-in-case marketing exercise to make Windows look more attractive to cross-platform developers while giving MS another strand to their bow when negotiating with Intel ?

3
1

There were never any big instruction set changes to the Alpha, once it was done it was done, later revisions just sped up the chips. DEC/Compaq fronted most of the money and half the engineering to make it happen, because their customers wanted it. It was far more than a marketing ploy, but once Compaq threw in the towel, there was no way Microsoft was going to shoulder all of the burden.

The speed challenges were always more about the crappy compiler, anyway; Microsoft's Alpha C compiler was worse than UNIX ones, and much worse than its x86 compiler. (Which if you've used VC6, is saying quite a bit!)

1
0
Silver badge

"But if nothing else it's nice to know that DEC Alpha lives on, albeit as an irritant, deep within the bowels of Windows."

Can't be the only one.

1
0
Anonymous Coward

More proof the recent "complete re-write" was a lie

Windows bug ridden old code with a new frock every few years.

3
4

Re: More proof the recent "complete re-write" was a lie

There has been no claim of a complete rewrite of Windows, ever. Or reason to - on a kernel level it's excellent (userland not so though...). Biggest changes were going from 2003/XP kernel to Vista/7, but this was mainly about improving scalability on large amounts of CPU cores.

2
0
bed

HAL

I could be wrong, but, I seem to remember that up to and including NT3.51 there was a Hardware Abstraction Layer between the OS and the CPU which mean the same code ran, slowly, on a variety of hardware. NT4 removed that and moved the GUI layer and only ran on x86 and Alpha - no longer MIPS. Quite a long time ago when we had a couple of 433 workstations and an Alpha server. Eventually ran Linux on one of the 433 boxes - went like stink!

0
1
LDS
Silver badge

Re: HAL

No. HAL is not a virtual machine which runs an intermediate code identical for each platform. HAL, kernel and user code are all compiled to the native CPU code of the target processors. The scope of HAL is to present a coherent API to the kernel for hardware access (i.e. physical memory management), thereby the kernel becomes more "hardware agnostic" and the OS doesn't require a specific kernel for each supported platform, only a specific HAL.

HAL is still present in Windows 10, AFAIK. Even on x86, for example Windows used different HALs for single and multiprocessor/core systems (the former didn't use some code to sync properly access to shared resources),

The fact that most of the graphic routines code was moved from user space to kernel (kernel, not HAL), has nothing to do with this. It was done because getting to and from the privilege ring 3 and 0 is costly (due to the protection checks, etc. etc.) , and slowed down performance. HAL is part of the kernel and runs at ring 0.

2
0

Re: HAL

The HAL basically encapsulates the stuff that's machine specific, not architecture specific. The kernel itself is still very much aware of the architecture, so it can do things like set up page tables, handle interrupts, and various other non-portable stuff.

Each component of the Windows kernel (Executive, Memory Manager, etc) has its own directory in the kernel source tree, and then for those that need it there are sub-directories for each architecture (i386, amd64, etc). Plus a small sprinkling of #ifdef's in the common code for special cases, though they are mostly about 32 vs 64 bit.

For x86 systems, the HAL is somewhat of a leftover from the days before the whole "PC" thing was decently standardized and there were lots of vendor-specific hardware in the box. In more recent times the big use-case was to support both ACPI/non-ACPI and APIC/PIC systems.

See https://support.microsoft.com/en-us/kb/309283

The kernel itself is also aware of uni- vs multi-processor systems, by the way; see https://en.wikipedia.org/wiki/Ntoskrnl.exe . The non-relevant locking is simply #ifdef'd out when compiling a uniprocessor kernel.

1
0

got his wires crossed somewhere

The article is ridiculous. The Alpha (even the original 21064) was perfectly good at bit-bashing and implementing any compression format you wanted.

Thanks to Anton Ertl, here are some timings of decompressing a 3 MB deflated file to 10 MB (i.e. gzip, zlib etc) using zcat:

0.351s 280m clocks 800MHz Alpha 21264B (EV68) 2001

0.258s 310m clocks 1200MHz ARM Cortex A9 (OMAP4) 2011

0.224s 240m clocks 1066MHz PPC7447A 2003

0.116s 230m clocls 2000MHz Opteron 246 2003

The fastest uses 25% fewer clock cycles than the slowest, and the slowest is not the Alpha.

Yes, the Alpha is the slowest in absolute time, but it's also the oldest and lowest clock speed. The Alpha was always considerably higher clock speed than its rivals made at the same time, at least until the Pentium 4 (which was high clock speed but not fast per clock).

What probably happened is Microsoft took some generic compression CODE (not format) that used byte reads and writes and simply re-compiled it for the Alpha, which didn't have byte instructions. It's not hard to rewrite the code to read 32 or 64 bits at a time and then use the (very good!) bit-banging instructions to split them into smaller parts. But that would take someone a week to do. You'd think Microsoft would have the skill and manpower to do that work, but apparently not.

The resulting code, by the way, would run faster on pretty much every other CPU too.

4
0

This does seem unusually pitiful in terms of The Register sensationalising Mr Chen's fine blog post then lots of blowhards wibbling away ignoring the point.

3
0

The DEC Alpha lives on in China ... apparently.

https://en.wikipedia.org/wiki/Sunway

Microsoft aren't alone in coding for the worst CPU on the planet :-)

2
0

" But that didn't stop DEC faltering and being acquired by Compaq in 1998."

I thought DEC took Intel to court for stealing the Alpha math co-processor and using it to upgrade the 486 to the Pentium. I thought they allegedly stole the plans when Microsoft gave it to them while they consulted on how to port NT 4.0 to the Alpha. Rather than force Intel to give DEC a few fabs to pay the penalties and royalties on the millions of Pentium, Pentium II, Pentium III's, etc, the judge allowed Intel to buy at least a large stake in DEC. As the new major shareholder of DEC, Intel settled the suit against itself, and then was required to sell off DEC piecemeal to avoid the antitrust implications to Intel. I heard that is what killed DEC.

0
0
Anonymous Coward

Re: that is what killed DEC.

Lots of things killed DEC. One day someone will write a worthwhile book on the subject (a relatively well known one already exists, but the things I read in it do not entirely tally with my experience of the company either as an outsider or an insider).

One of the biggest things responsible for the death of DEC went by the name of Robert "GQ Bob" Palmer. You won't find much written on the subject, but this snippet may be illuminating:

https://partners.nytimes.com/library/tech/98/09/biztech/articles/10digital.html

And when he wasn't selling (or giving away) the company's future to competitors in software, he and his successors were doing it with DEC's hardware.

What the chip and system engineers knew:

http://www.cs.trinity.edu/~mlewis/CSCI3294-F01/Papers/alpha_ia64.pdf

What Palmer was doing at the same time was agreeing a ten year deal as part of a court case settlement with Intel [1] to abandon Alpha in favour of the future "industry standard 64bit" architecture, IA64.

Water under the bridge, sadly.

[1] https://www.cnet.com/uk/news/intel-digital-settle-suit/

2
0

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Forums

Biting the hand that feeds IT © 1998–2018