back to article Windows 7's dirty secrets revealed

While chief technology officer Ray Ozzie was away in the clouds at Microsoft's Professional Developer Conference, technical fellow Mark Russinovich got down and dirty with the true heart of Windows - the kernel. He presented a two-hour session on changes made to the kernel used by both Windows 7 and Server 2008 R2, shedding …

COMMENTS

This topic is closed for new posts.

Page:

  1. Valerion

    Russinovich

    I was worrried (well, it wasn't exactly keeping me up at night, but you know what I mean) when MS bought Sysinternals. Russinovich was obviously a very talented guy and the products were great, but I thought he'd be buried somewhere deep in MS and forgotten about.

    But it seems he is right up there, and doing lots of work on the core parts of Windows. Frankly, that's a GOOD thing and gives me hope for the future.

  2. peter 45
    FAIL

    Crashing

    I still have not got over the fact that a corrupt .doc file will not just crash the Office application, but the entire Windows OS. How can, what is essentially a data file, do that?

    Oh...wait. It is Microsoft we are talking about.

  3. A. Lewis
    Thumb Up

    Good stuff

    I saw two of Mark Russinovich's sessions at Tech-Ed. That man knows an insane amount about Windows! It's impressive not only how much he knows but that he still manages to get little digs at MS into his very well delivered talks.

    Also, seeing him have problems with his VM so run his 'tests' on the machine running the presentation resulting in it giving random blank error boxes throughout the rest of the session was funny indeed.

  4. Geoff Mackenzie

    We don't really understand those dependencies

    I know this has already been said, but wow, that's incredible.

    Also, MinWin is a misnomer. Most of my machines run less Windows than that.

  5. Peter Kay

    @Dan55 et al, about re-architecting Windows

    Microsoft couldn't even if they wanted to. They're limited by their userbase, application and driver writers.

    You can debate whether Vista was a good product (buggy on release, fixed later is my view), but the real whinges came from a failure of non Microsoft people to do their job properly. Crap drivers, inability to run as non superuser etc. The OS /was/ re-architected and people didn't like it.

    Workarounds like FTH are a pragmatic solution for the userbase. Microsoft simply cannot take the OpenBSD attitude of enforcing a strict heap, especially since as soon as OpenBSD did so it broke a fair amount of software including some very old programs. OpenBSD is a great OS, but its target market is considerably different to Windows.

    I would also note, before anyone gets too superior, that whilst Windows may be gnarly, even some of the more pared down Unixes are not entirely free from the malaise of unexpected dependencies. They may not have quite so many shims to support misbehaving applications, but there are odd or unexpected behaviours, the origin of which are pretty much lost in time.

    I installed Windows 7 last night, it found almost all hardware (29160 : use Vista drivers, Audigy : online drivers) including setting up my four monitors correctly. Finally, it also suspends correctly, which is something Vista never managed..

  6. Bassey

    Re: Dan55

    "It's as if MS are back where Apple were in the late 90's...They know the OS needs an overhaul, but are bodging the overhaul as well instead of re-architecturing the OS from the ground up."

    What a pointless argument. Apple were a nobody with a nothing footprint - particularly in the business world. The effect of "re-architecturing" <shudder> was minimal because hardly anyone was (is) using it. The loss of backwards compatibility only affected a tiny number of people and they were mainly home users.

    For MS to re-build Windows from the ground up and lose backwards compatibility would destroy the company. The reason people upgrade their Windows OS is because they perceive (right or wrongly) that they are gaining extra benefits/features whilst maintaining the set of applications that they already have. In other words, their past investment is not wiped out.

    I know this is a hard concept to grasp for your average mac user but BUSINESSES invest millions in software for windows. They are not going to write that off overnight because MS want to re-build windows from the ground up for architectural reasons. The MS CEO who makes that call will get laughed at, then sacked.

  7. John Sanders
    Flame

    I'm probably the only one here....

    I too have a lot of respect for Russinovich, the guy has proven time and time again to be awesome.

    But I'm sorry to say that he now works for "The man" now, BEFORE he joined MS he could claim integrity on his technical claims, not any more.

    As a MS employee he won't stand up for the public when some big corporation decides to cooperate with MS on the next commercial spyware/rootkit scheme like the Sony rootkit fiasco.

    He will stand aside, or simply say and do what the man tells him to say or do.

    Vista/7 has a mechanism a type of restricted/reserved process that in theory, only MS can control, those processes oversee the DRM and other "undocumented" stuff.

    And now that he's working for MS god knows what else he is cooperating on with MS, or what is he helping them to hide.

    I'm being paranoid? Hell yes, I do not trust the "big", plain and simple. In this time and age when anyone with a budget is making decisions in the name of my safety or well being, all they try is to provide themselves with the means to remain in power.

    That leads to totalitarianism, and MS is like that, it's been proven again and again.

  8. Anonymous Coward
    WTF?

    @Eddie Johnson

    Just because I feel like bitching about something this morning I guess...

    Like NT4 that much? Do you? Had to work on it much? Stable huh? When's the last time you booted an NT4 server that wasn't a fresh build and it didn't say "At least one service failed to run at startup"? Oh, I'll just put that missing .dll on my flash drive and... Oh, wait...

    I do love when you boot a later version of Windows and it claims "Built on NT technology" Isn't that like entering your "PIN number"?

    Oh well, if it weren't for computers and their many failings I wouldn't have a job that I can at least stand most of the time, so I guess I'll shut up now.

  9. WinHatter
    Pint

    M$ workaround

    Interdependencies not well understood.

    Well let's have all of it in 1 single big fat DLL ... problemo solved.

  10. Anonymous Coward
    Anonymous Coward

    So I've been right to disable UAC...

    'So what is the point of UAC? "It is about one thing, which is about getting you guys to write your code so that it runs well as standard user".'

    Ah. So if I, as an end-user, choose to disable UAC, it won't do me mr my system any harm. Because, according to Microsoft itself, the purpose of UAC is not to confer any security benefits, but just to force developers to write better (and more secure) code.

  11. mittfh
    Linux

    Best workaround

    Bill's fan club will probably hate me for this, but it has to be said:

    1) Download a Linux Distro

    2) Burn to CD/DVD (Mandriva Free is a DVD ISO)

    3) Insert CD in CD drive

    4) Reboot

    5) When prompted, say "Yes, I do want to reformat my HDD"

    Of course, it's not quite as simple as that. You'll probably want to backup your documents first. And in between steps 4 and 5:

    4a) Realise your computer has ignored your CD and is starting to boot Windows.

    4b) Reboot again

    4c) Hit Delete like a mad thing to get into your BIOS

    4d) Spend a merry 5 minutes working out where they've hidden the boot order screens.

    4e) Amend your boot order.

    4f) Press F10 (usually corresponds to save and exit)

    4g) Wait for the computer to reboot and start loading the CD

    4h) If it's a Live distro, double check that most of your hardware still works.

    4i) Double click the button to install it to your computer.

    -oOo-

    But I'm not surprised at the Microsoft employee admitting they're not entirely sure of the dependencies any more. What almost certainly happens when they're building a new version of Windows is to start with the existing version, then change / tweak bits as needed. And if the programmers don't document their code fully, then seek alternative employment, you're screwed. Heck, there's probably still some Win 95-era code hiding in Win 7... and maybe the occasional method or two from Win 3 or earlier...

  12. Rob Davis
    Thumb Up

    How refreshingly pragmatic

    Keep up the good work, Russinovich.

  13. Ron1
    Alert

    30% compared to What?

    "Memory footprint was reduced by up to 30 per cent"

    Compared to what? Vista, XP, windows 3.1, DOS?

    30% is relative; in itself it tells nothing if not stated compared to what.

    Don't get me wrong; I think Mark ruleZ! Microsoft was lucky to snatch him from his Wininternals/Sysinternals. I am sure he mentioned what he was comparing and the statement is just taken out of context.

  14. John Square
    Pint

    @Ed Courtenay

    "the worst software I've come across tends to be in-house applications that were thrown together by some office junior while on work placement five years ago, that inexplicably become vital to the operation of the company (although not critical enough to employ anyone to code it properly)."

    Very good point, sir. It's true that MS have probably cocked up by trying so hard to maintain backward compatibility over the years, but a chunk of the blame must be laid at the corporate IT world, who allow bag o'shite apps to slowly become mission critical, when originally they were coded to be stop-gap measures. With all of the Freeform DYnamics stuff on the reg in the last couple of weeks about IT Governance fresh in my mind, you've highlighted a problem there that doesn't get talked about often enough: IT departments should have the balls to tell the business to take a running jump when the business comes knocking to demand a technical fix to a failure of management.

    What's that? Your MS Access-based app runs poorly, and you absolutely cannot do without it? Stop using it, and go find a proper app that runs on a modern OS and migrate now. Don't postpone another six months, or to the start of the next financial year, 'cos the situation will just be worse then.

    "If this application breaks because of a new operating system, guess who gets the blame? The office junior? No, Microsoft." It should be the director of the department responsible.

    <As you deserve one.

  15. chrisj1
    Gates Halo

    We don't understand these dependencies??!!

    "We don't really understand those dependencies" ????!!!!

    My god what is wrong with this guy(s)?? Those MSFT developers!!!! No wonder their OS is such a MESS!!

  16. Jason Bloomberg Silver badge
    Thumb Up

    Dependencies

    "We don't really understand those dependencies"

    That's no surprise and refreshing honesty. I've seen far smaller projects than "Windows" get out of control, been there, done that, got the T-shirt, and lived to regret it ...

    Someone thinks it would be good to add encryption into a simple module which reads config files when the line starts with a certain 'flag', and it's great, works well doesn't break any compatibility, adds security for any app which wants it, everyone is happy, users, developers, customers, product reviewers, management.

    Down the line though anyone who uses the config file module also has to include the encryption module and all that relies on ( and there may be circular dependencies ). Now try to work out exactly what has which dependencies and it's a nightmare, even more so trying to prune them down without breaking anything.

    It's easy to say that Microsoft shouldn't have got themselves into this mess ( and I'm sure they'd agree ) but let's be more honest and realistic than that. Seemingly minor and inconsequential changes can have massive impact down the road and such problems can afflict any software development. Microsoft are also at a disadvantage because users expect backwards compatibility, so it isn't just a job of re-doing it properly but also an often necessity of 'bloat' to do so and maintain compatibility. It's a long, hard slog and an almost impossible 'win-lose' situation.

  17. Wilko
    FAIL

    Proof if proof be need be

    that MS still haven't got a sodding clue what they're doing.

  18. James Hughes 1

    Why are people so surprised about the dependency issue?

    Any C developer who has worked on a large code base, written over a number of years, by a number of different people, will know that dependency problems are pretty much par for the course - DLL and header files both.

  19. Jax 1

    "we don't really understand these dependancies"

    Why is everyone so very shocked by this statement? I don't fully understand the entirity of the C# code base we have here (I know the majority of it) and people call me the systems architect.

    Our code base is just an app that runs on WinCE, I would imagine that Windows itself is much, much, much more complex than this.

    It would probably be a hell of a mindfuck to place that entire model into your head. Any automated diagram you produce would be insanely hard to read as well. Have you people developed large and complex software? It's hard you know, _really_ hard.

    The problem exponentually grows and Windows is huge. So i'm really not surpised. I heard when building Vista each developer was about 6 branches away from the trunk more or less at a minimum. It would take about 6 months for your changes to hit the trunk.

    Linux/Unix doesn't have economic pressures, Windows histrorically has. Therefore there are a number of WTFs in the Windows code base. This is what happens when you need to ship by a deadline and on a budget.

  20. Annihilator
    FAIL

    @Dazed and Confused (and other FTH whiners)

    "User mode crashes? How the hell do you crash an OS from user mode?"

    Yes, Dazed and Confused indeed. Who said the OS crashed? How often do you see a Windows box crash these days? I can't remember the last time and I run mine 24x7. He was talking about APPLICATION crashes. Poorly written apps that fall over and die because they're a stinking pile of crap.

    What happens when such an App exists? User complains to software manufacturer, who says "It's Microsoft's fault". Enough bad coders out there make MS the common link. So MS have decided enough is enough, if they detect you're incapable of doing your own memory management, they'll do it for you.

    Maybe if they get very good at detecting this stuff, they can just print a big error message saying "software manufacturer A is crap, we suggest you get a refund. Here is a list of their competitors" and remove FTH

  21. Will 3
    FAIL

    @ FTH commenters

    Talk about missing the point, the lot of you. The fact that it automatically stops nannying your code when run with a debugger is the best part. Maybe if you had actually tested your code before deploying it then complaining that windows is making it crash less wouldn't be an issue.

    *rolleyes*

  22. Anonymous Coward
    Anonymous Coward

    Can I buy MinWin on its own?

    Don't want all the other shite that comes with Windows. Ta.

  23. Filippo

    @Annihilator

    Right on. And mr. "Dazed and Confused" isn't even the only one. It seems many of these rabid anti-Windows fanatics haven't actually used any Windows after 98 and still believe that Windows is prone to crashing.

    While we're sticking to prejudices that were true 10 years ago, I guess it will be fine for me to keep saying that Linux is exceedingly hard to install and configure, and that Apple computers are severely lacking in apps?

  24. Sean Timarco Baggaley

    Rewriting Windows is possible...

    ...because it's been done before. The MS-DOS-derived Windows 9x/ME series was a very different beast to Windows NT and 2000 under the hood. So it IS possible to do a rewrite of the kernel and its supporting libraries. The trick is to retain backwards-compatible APIs. What those APIs actually do needn't be the same under hood, as long as the end results are.

    (Apple isn't a good example; they sell to the consumer sector, not corporates, so they had far fewer issues with legacy apps.)

    As others have pointed out, running XP in a VM is likely to be fine for most corporates. MS could easily include a suitable VM in some of their various Corporate Editions of New Windows. (It's probably not worth including in consumer editions.) This frees MS to make the bold choice of building a brand new OS from the ground up.

    And yes, we really DO need an alternative to OS X, *BSD and Linux. Those OS families are fundamentally UNIX variants and have 30-odd years of legacy cruft and design in them too. While this doesn't mean they're unstable or bad, but UNIX's design heritage means it inherently limits the evolution of software design and development, not to mention UIs. (UNIX was designed in the age of punched cards, paper tape and big, reel-to-reel magnetic tapes. User interaction in applications was minimal at best.)

    Windows isn't much younger, and the less said about GNU's Hurd project, the better. We need fresh approaches better suited to the 21st Century's needs.

  25. Andy Enderby 1
    FAIL

    echoing many others

    posted by Joe User - Russinovich admitted: "We don't really understand those dependencies".

    If Russinovich is finding it impenetrable, then Microsoft have really screwed the pooch.

    Windows 8 needs to be a properly engineered, ground up, re design. Anything else is unacceptable.

    Win for Russinovich, his honesty is admirable, fail for MS.

  26. /dev/null
    Boffin

    "the GUI-free Server Core edition"

    If you've ever seen a screenshot of Windows Server 2008 Core, you'll realize it's *not* GUI-free at all! - they've just stripped out the Explorer shell and *most* of the GUI apps, but kept things like Notepad and Task Manager. Pretty pointless really.

    Have a look here:

    http://www.petri.co.il/understanding-windows-server-2008-core.htm

  27. Vanir
    Thumb Up

    Well done Russinovich

    I love these comments! Great entertaining stuff: thank you all.

    Understanding 'dependencies':

    I've been doing 'professional' software development since 1997 and I don't think I fully understand the dependencies of a C/C++ 'Hello World' program. Most coders I have worked with do not even consider coupling never mind dependencies for each and every context of the problem / solution domain that they are dealing with.

    I feel that MS has been afflicted with the same disease as almost every other company that develops software. There is a saying in industry and especially software that goes like this 'on time, within budget, good quality; choose any two'. Well, I think the disease is that sales and marketing are just choosing on time, their time. It is their time objectives that has priority. They must have their bonuses.

    I like honesty, don't you? Russinovich has done good. I'll listen to him but I won't listen to Balmer.

  28. Ken Hagan Gold badge
    Flame

    It's the (stupid) developers, stupid!

    Two points really stick out here.

    Firstly, there are apps out there that wouldn't have worked if the version had been 7. WTF?! For twenty years, each new OS from MS has been plagued by UTTER FUCKWITS who can't write >= instead of ==. Note the present tense here. These apps would have broken *before* version 6 if it was just a matter of linking in old code from somewhere. Someone is actively writing *new* code that commits this error. And if Microsoft do the right thing and let the app do the wrong thing, Microsoft get the blame.

    Secondly, that "developer concern" that the fault tolerant heap might hide bugs. Er, yes, that's the point. Microsoft *want* to hide bugs (added by stupid developers) from innocent end-users. I'm sure end-users will applaud MS for this. If "developers" are "concerned" about such a plan, perhaps they could pull their fingers out and start catching buffer overruns *before* they ship. The tools have been part of your IDE for a decade or more. Try using them you clueless pratt, and stop criticising MS for protecting the end-user against your moronic quality assurance.

    Much as I hate Microsoft, I hate FAR, FAR more the idiot developers who pour bugs into the ecosystem and force MS to make each new version of Windows even slower than the last. (It's not like MS need the help guys.)

  29. Annihilator
    Boffin

    @ Sean Timarco Baggaley

    "Rewriting Windows is possible ...because it's been done before. The MS-DOS-derived Windows 9x/ME series was a very different beast to Windows NT and 2000 under the hood"

    Indeed, they were very different. Which is why they ditched one and extended the other. They didn't drop everything and start again. They extended the NT/2K codebase. Hence the versions - Win 2K built on kernel 5. XP was 5.1.

    To throw everything out and start from scratch would take YEARS, and they'd run out of money before it finished. What they've done over the releases to date is tightened up the core - aka the kernel. Get that stable and it shouldn't matter what fails on top of it. They're now extending that to tighten up "the experience". They could have done it the Apple way - the iPhone's version of FTH is "don't let shoddy stuff run at all" aka, the App Store.

  30. Ken Hagan Gold badge
    Pint

    A way out

    Microsoft should make the next version of Windows free (as in beer, see icon). Then they should charge per shim.

    Customers would then have a real incentive to bash ISVs over the head about quality. The worst offenders appear to be in-house applications, but corporate customers have the deepest pockets, so it serves them right. The average Joe who only wants to use a browser, email and media player wouldn't have to pay a penny.

    MS get the advertising benefit of rock-bottom pricing, plus a guaranteed revenue stream from all the dickheads who have caused them so much grief for the past couple of decades. What's not to like?

  31. alyn
    FAIL

    Not good enough

    When Minwin will run of a 1.44MB floppy, then I will be impressed. Linux can do it (Toms root & boot), though admittedly without X Windows.

  32. Dave_H
    WTF?

    Who can't manage large code bases?

    WTF - I have 11 million lines of code, spread across 3 products, each with a differing architecture. I have no design documentation, but they were built with good architectural and software principles. I manage to support (and bug fix where required) these by myself because they were properly written. Mind you three years of doing this has screwed my brain a bit!

  33. Anonymous Coward
    Thumb Up

    A floppy ?????

    You can get QNX on a floppy?!?!

    Really?!!?

    Wow!

    I'll just go and find a PC with a floppy drive. . . . oh, wait. . . .

  34. Eddie Johnson
    Grenade

    @@EddieJohnson

    >>Like NT4 that much? Do you? Had to work on it much? Stable huh? When's the last time you booted an NT4 server that wasn't a fresh build and it didn't say "At least one service failed to run at startup"?

    Its been about a month. And before that it was about 15 months. Why? Because my NT servers are booted less than once per year. They are stable, why would a DLL go missing? Why would I want to use a USB Drive? Ever consider that allowing USB drives is WHY your machines are unstable? Convenience is NOT the number one priority.

  35. ZenCoder

    There are not easy answers.

    Any massively complex software engineering project is going to be too complex for any one person to understand and even with the most talented people in the world developing it, at some point the design is going to get messy.

  36. rhdunn
    FAIL

    @Will 3

    Great, you now have an application that behaves differently when run by a user/tester to when you are running it in the debugger.

    I see at least two fundamental problems with this:

    1. testers/users may initially see the application crashing, but when they try and reproduce it, the issue magically disappears -- great for consistency and reproducibility!

    2. when debugging the application to investigate a different issue, the application starts randomly breaking.

    Welcome to the Microsoft uncertainty principle!

    Here's another question -- what happens when drivers (e.g. NVidia) start crashing, where does FTH apply then (to the kernel? on kernel memory?).

  37. Anonymous Coward
    FAIL

    @Dave_H

    "I have no design documentation, but they were built with good architectural and software principles."

    Wouldn't software principles include design documentation? It does in every software development I've been involved in. Or did you design it as you wrote it?

    Either way - one man supporting one man's code is not impressive.

  38. dr_forrester
    FAIL

    WTF?????

    How do they "[not]... really understand those dependencies". This is basic project management. Shouldn't there at least be some sort of diagram or something somewhere that MUST be added to each time a new module or dependency is created?

  39. Steve B
    IT Angle

    Wrong approach

    If the OS detects that a program keeps failing it should simpy delete it, thus serving 2 purposes.

    1-Automatically Removing crap software

    2-Educating the programmer who will very soon get hacked off with having to rewrite and may start to get it correct first time or even give up.

    Unless, of course, they are MS programs that keep failing!

    Meanwhile with all this kernel and user stuff interacting incorrectly, they should have invented some sort of access control level (ACL) and combined that with an access control register (ACR) to see if the code currently has the correct privileges to execute.

    Even simpler have a flag that states code or data and make all data unexecutable.

    I'm pretty sure the patent is about to run out on good operating system design, so MS should be able to reinvent it soon.

  40. John Smith 19 Gold badge
    Happy

    A note on automatic diagramming

    while the dependancy diagram might well be huge tools exist to detect cycles in such graphs and report them.

    Figuring out how to break the cycles (without breaking any existing software) will call for substantial design skills.

  41. alex d 1

    MinWin only solves post-XP problems

    Windows Embedded is already everything that MinWin is supposed to be (like running in 40MB ram, _with_ GUI). However, Windows Embedded is still based on XP. MinWin is simply the attempts to undo the damage of the Vista development era (where software quality took a nosedive) and make feasible a Vista/7-based version of Embedded.

    This is the angle the article missed.

  42. John Smith 19 Gold badge
    Coat

    Those who know s^&t about history

    are going to repeat it.

    Again and again.

    Core concepts devised by a small team -> Unix

    Core concepts slapped on by any developer to give ultimate-do-anything-OS -> OS360 /

    Windows

    Mine will have a copy of "The Mythical Man Month" in it.

  43. Anonymous Coward
    Flame

    Those who don't understand UNIX are condemned to reinvent it, poorly.

    Unix may be 40 now, but it remains a shining light of software. Just finished my Lion's Commentary on Unix v6 from 1976 which is 4 years before I was born. Amazing book! What is amazing is how much hasn't changed. So many of the ideas have survived because they are good. Some retro fitting of things that came after the original design weren't done in a way that fitted, but when the design was truely modernized, i.e Plan9, it didn't take off. Though some ideas did make it into other Unixs.

    Windows belongs in the bin. It's too complicated, too tangled, too limited, too grown by sales teams, too closed. As a OS, it's of no interest to me anymore.

  44. J-Wick
    Grenade

    mittfh

    Sorry mittfh, just wanted to explain why I voted you down.

    (1) Installing any OS is never that simple, be it Windows, Mac or Linux

    (2) You ignore the 'applications and data' side of computing. People use computers as a means to an end, not as something to show off their OS. By ignoring the realities of computing, not only is your entire post worthless but you come across as naive.

    Sorry.

    (Ubuntu + XP VM under VirtualBox user)

Page:

This topic is closed for new posts.

Biting the hand that feeds IT © 1998–2019