back to article Systemd kills Deb processes

Debian's systemd (system daemon) has acquired a default config that nobody likes: it kills running processes on logout. The result, as detailed in a bug report that surfaced on Slashdot, is that long-running processes are killed. Since a capable Linux user would treat this as normal behaviour – why sit there watching a screen …

  1. FrankAlphaXII Silver badge

    It won't last unless Lennart and Co. decide its a feature, not a bug.

  2. Will Godfrey Silver badge
    Unhappy

    Hmm

    So feature creep continues. Why am I not surprised.

    I think this thing should be renamed 'Mold'. No matter what you do, it slowly expands degrading every thing it touches.

    1. Doctor Syntax Silver badge

      Re: Hmm

      "I think this thing should be renamed 'Mold'"

      Maximally obfuscated Linux daemon?

  3. Tom 64
    Coffee/keyboard

    Broken expectations

    Yup. I frequently push processes into the background as an unprivileged user and immediately log out expecting that process to stay the fuck alive until I choose to terminate it.

    Having systemd kill processes on user log out may well have its place on grandma's or little Johnny's laptop, but it would irritate the crap out of me anywhere else.

    1. Voland's right hand Silver badge

      Re: Broken expectations

      This means you never had to work on an "adult" unix.

      Unless your friendly local sysadmin turned job control off, SCO, AIX and other full spec Sys V + Posix implementations would all kill your processes on logout.

      1. Jan 0

        Re: Broken expectations

        System Vile or not, you'd 'nohup' them to keep them running when you logged out

        1. Paul Kinsler

          Re: you'd 'nohup' them to keep them running when you logged out

          Indeed. Only if systemd is killing nohup'd processes is it really misbehaving IMO (although I accept that modern expectations may differ, and notwithstanding systemd's other misbehaviness).

          1. ckdizz

            Re: you'd 'nohup' them to keep them running when you logged out

            I think this is exactly the point. While it's not immediately desired behaviour, it's eminently sensible behaviour and should be expected if you're trying to run processes as an unprivileged user. After all, most systemd processes when started from unit files are daemonised, even those run as unprivileged users. And that's the way the (terrible) systemd manual says they should be run. I'm going to guess the problem is somewhere in the middle of Polkitd and systemd.

            I can see there's a need for it, and I'm a Debian user, but personally if I have background processes to run that I need to stay running I make sure they're going to carry on running after I log out. That's probably why I haven't encountered it.

            I'm on the fence as to whether or not this needs to be fixed or the requirement for a persistent process needs to be specified in some other way. Otherwise users could be left with a bunch of processes running simply because they didn't explicitly stop everything in logout.

    2. Alan Brown Silver badge

      Re: Broken expectations

      "I frequently push processes into the background as an unprivileged user and immediately log out expecting that process to stay the fuck alive until I choose to terminate it."

      And _I_ frequently have to deal with runaway processes eating 100% cpu because some fucktard has backgrounded it and logged out, instead of leaving the process attached to a "screen" session.

      A lot of sysadmins on shared systems will be applauding this option. It saves us having to find and kill misbehaved software processes (Mostly IDL and emacs) left behind by people despite being told not to do it.

      If everything was well-behaved then it wouldn't be a big deal, but badly written software (especially "scientific" stuff written by commercial entities) is just as prevalent on *nix is just as on windows - and the "fix" is usually the same - "fix one minor bug, ignore most of the rest, add 10 new unnecessary features, charge for an 'upgrade' "

      1. DainB Bronze badge

        Re: Broken expectations

        You might soon find your smarter users running their shell sessions as "system-run bash" to circumvent your restrictions. I'm sure you'll enjoy it.

  4. nematoad Silver badge
    Unhappy

    Inevitable?

    "Systemd kills Deb processes"

    People kept warning that this sort of thing might happen.

    Systemd has too many fingers in the Linux pie. Whatever happened to "Do one thing and do it well"?

    1. John Crisp

      Re: Inevitable?

      "People kept warning that this sort of thing might happen."

      Yup. Unfortunately 'bling' and RHs decision to try and take over the world using profit via obscurity prevailed over common sense.

      "Systemd has too many fingers in the Linux pie. Whatever happened to "Do one thing and do it well"?"

      The baby got thrown out with the bathwater.

      1. Tom 7 Silver badge

        Re: Inevitable?

        "The baby got thrown out with the bathwater."?

        More like they used a pressure washer to clean the Sistine Chapel.

        There may have been some good points behind the original thinking about Systemd but they were forgotten or made redundant years ago.

    2. MotionCompensation

      Re: Inevitable?

      Actually, systemd does only one thing and it does that well: killing the principle of doing only one thing and doing it very well. I hope I've now made it disappear in a puff of logic...

    3. Down not across Silver badge

      Re: Inevitable?

      Systemd has too many fingers in the Linux pie. Whatever happened to "Do one thing and do it well"?

      Lennart happened.

  5. sysconfig

    Creating problems that didn't need solving

    That's, in a nutshell, systemd.

    If distros think it's useful to speed up boot times and respond to device changes (wifi, GUI interaction etc) on desktops and laptops, that's one thing. But it has no f***ing business on servers. The average server's POST time is probably around 2 minutes due to controllers etc. I don't give a flying f*** if Linux takes 10 or 30 seconds to boot after that. It only happens once every blue moon anyway (kernel & glibc updates), and while it reboots, another server is taking over the job.

    Systemd is the idea of a self-centred hobby kernel developer (now unfortunately sponsered/employed by RH), who shows complete disregard for real world problems.

    It messes with things it shouldn't (leave them processes alone!), adds layers and layers of unnecessary complication to simple tasks (who asked for binary logs, please? if you store logs *only* on the server, and not remotely, you have bigger issues than log integrity), and the list continues.

    Just keep those screw-ups coming. Makes it easier for me to propose FreeBSD to clients.

    Rant over - until systemd messes up yet another thing, which won't be long.

    1. Gerhard Mack

      Re: Creating problems that didn't need solving

      The part you are missing is that the change was mainly because the old system broke horribly and required manual intervention when you ended up with an even moderatly complicated server setup (Fiber Channel, iSCSI, Distributed Filesystems etc)

      This latest change on the other hand, is very desktop oriented where Xservers never seemed to clean up after themselves properly and thankfully it has an off switch which and I will be spending the next few days turning this off on my servers and on for my desktops..

    2. Alan Brown Silver badge

      Re: Creating problems that didn't need solving

      "I don't give a flying f*** if Linux takes 10 or 30 seconds to boot after that"

      Some of our more complex systems take in excess of 20 minutes to fully restart, thanks to singlethreaded startups and clustering races needing to be avoided across FC mesh. One system was noted for taking 24 hours to start, thanks to singlethreaded FSCKs of a few hundred TB (and SysV netfs scripts which needed heavy beating to multithread the checks across FC mesh - yes, FC is regarded as a network and yes they force all network checks to be single threaded)

      The Systemd approach has the right idea by parallelising as much as possible, but it's a Frankenstein Monster in a lot of areas and you have to understand the dependency chains if you're going to tweak with it.

      Incidentally, it's NOT a huge monolithic process and can be understood _if_ you take the time to do so.

      FWIW I spent more than a decade resisting SysV over BSD rc startups on Linux then spent months kicking myself after taking the time to understand the differences. systemd has a lot of positives for large systems, the biggest negative being Lennart himself.

      The blind hate is unjustified. You might not want to use it on your single-user box and that's fine but in a large, complex, multiuser environment it's a different story. That said, it's far closer to an ideal startup sequencer than BSD or SysV were and _that_ is why it's not going to go away until something better comes along.

      As others have pointed out: Unless you nohup your processes, most *nixes will shut them down when you logout. This is posix behaviour being enforced - and as it's controllable in security configuration it's not a bug, simply a change in defaults to "what should have been" all along.

      1. DainB Bronze badge

        Re: Creating problems that didn't need solving

        "The Systemd approach has the right idea by parallelising as much as possible"

        Considering that my RHEL7.2 VM takes about 5 times longer to boot than RHEL6 I'm not sure what parallelizing you're talking about but it obviously does not work well.

        "As others have pointed out: Unless you nohup your processes, most *nixes will shut them down when you logout."

        Yes, that's an expected behavior, since circa 1973. Killing nohup-ed processes - not.

      2. rhsoft

        Re: Creating problems that didn't need solving

        > As others have pointed out: Unless you nohup your processes,

        > most *nixes will shut them down when you logout. This is posix

        > behaviour being enforced

        a lot of text without any clue - otherwise you would have realized that it kills nohup-processes, screen and tmux too as long they are not running as service or started with sytemd-run in a own scope

        and now you!

      3. sysconfig

        @Alan Brown - Re: Creating problems that didn't need solving

        Incidentally, it's NOT a huge monolithic process and can be understood _if_ you take the time to do so. [...]

        The blind hate is unjustified. You might not want to use it on your single-user box and that's fine but in a large, complex, multiuser environment it's a different story. That said, it's far closer to an ideal startup sequencer than BSD or SysV were and _that_ is why it's not going to go away until something better comes along.

        Firstly, it's not blind hate, but my personal experience and opinion, which I am entitled to.

        Secondly, more importantly, do not make any assumptions about the environments I am in charge of or my willingness or ability to learn systemd. You can keep your Lennart-like attitude to yourself.

  6. hplasm Silver badge
    Holmes

    I haven't noticed this yet-

    Oh wait, I moved to Devuan...

  7. Ken Hagan Gold badge

    Apparently the show's over

    A comment near the end (at time of writing) of the bug report states that the change has been reverted already, so there is no story here anymore.

    I guess that's why they call it "unstable".

  8. g00se
    WTF?

    Get me Hennimore!

    Since a capable Linux user would treat this as normal behaviour – why sit there watching a screen when there's nothing happening, it's unwelcome.

    I must have read this a dozen times. And it still doesn't make a blind bit of sense ,,,

    1. FrankAlphaXII Silver badge

      Re: Get me Hennimore!

      I think they mean that unless the process is nohupped it should be getting killed at logout, as that's what hup does.

      As Ken pointed out above though, its been reverted so its a non-issue now.

  9. jake Silver badge

    That's the entire problem in a nutshell ...

    ... the entire systemd cluster fuck assumes it knows more about how I use my systems than I do. That's a neat trick, considering that I've been using *nix longer than any of the authors of systemd have been using computers.

    BSD (servers) & Slackware (desktops). Try it. You might like it.

    1. Anonymous Coward
      Anonymous Coward

      Re: That's the entire problem in a nutshell ...

      * weren't born when I started using computers

    2. FrankAlphaXII Silver badge

      Re: That's the entire problem in a nutshell ...

      BSD ain't bad on desktops either. PC-BSD and GhostBSD are actually rather good. And its closer to "real" UNIX.

  10. Anonymous Coward
    Anonymous Coward

    A system daemon closes when a user logs out.

    No matter how many times I try to work out that logic I just can't see the reasoning.

  11. x 7 Silver badge

    "........ it kills running processes on logout.

    The result, .......... is that long-running processes are killed."

    I've heard of stating the obvious, but that is a bit extreme

  12. Anonymous Coward
    Anonymous Coward

    Introduced by Red Hat ...

    Red Hat introduced this "feature" in RHEL 7.2 and ignored the bug report that pointed out it was killing processes off.

    During the testing of Oracle Linux 7.2 this feature was found to be killing off the database. Just login as oracle and logout again and boom, no more database. Not exactly a desirable feature in a server.

    The guilty option was identified and the default config now disables it. Systemd does some bizarre things and really needs its wings clipped. There are a few good things in there but there's an increasing number of things, like this, that simply make no sense. Systemd is rapidly turning a teriffic OS into something totally unusable.

  13. aloev

    Systemd is not a Debian invention

    Although every major Linux distribution (and thus also Debian) has jumped onto the Systemd train, its mainly RedHat 's Leonard Poettering who drives the development.

    For those who care about the freedom of choice regarding their init system: There is a growing number of developers who support the Devuan project, a future full fork of Debian, which is currently in beta status:

    http://www.devuan.org

  14. ckdizz

    I love systemd discussions on El Reg. It reminds me of hearing my grandad talk about how metric bolts were going to end the world.

    1. Adam 52 Silver badge

      Possibly a better analogy would be complaining that plastic rivets were replacing bolts in the engine because they look good on the trim panels.

      My Dad used to complain about everything modern being glued and rivetted with no thought to serviceability, seems a similar argument.

      1. ckdizz

        It's not a similar argument at all. I can write a unit file without breaking systemd. If you can't, then the problem isn't with systemd, because I'm a veritable suckfest at my job. So if you're routinely breaking systemd, I'd look elsewhere for the source of the problem. Also, systemd is actually entirely modular. So if you build it from scratch, to a good extent you get to choose what you want to integrate. Arch's systemd is way different from Debian's, for example. Hell, it's even more advanced than Fedora's.

        The difference is in the tools you use to do the job. My grandad's hatred of metric was based entirely on the fact that the Germans and Japanese used metric fasteners, and that he had to buy new tools to use the job - his AF were less used, and his Whitworth were almost entirely obsolete by the 80s. People's hatred of systemd is that Poettering is an author, Red Hat have championed it, they have to learn something new and their knowledge is becoming obsolete.

        If some people had their way we'd be using a bicycle powered smoke signal generator instead of a phone.

        (As an aside, plastic fastenings in engines are extremely uncommon, as the heat and vibration in an engine will destroy them fairly quickly. If plastics or resins are used, it's usually as an adjunct to metal, or in components where it's necessary for strength and security to replace the fastener after it's been taken apart. Your dad was, in a sense, right about things being glued or riveted more these days, but that's largely because the ability to service complicated parts that started appearing in cars in the 80s and 90s has meant that even experienced mechanics don't have the tools to work on them - the dual clutch solenoid units spring immediately to mind. Nevertheless, the source code for systemd is available for you to hack away at all you want.)

        1. Tom 64
          FAIL

          "Also, systemd is actually entirely modular."

          So that's why its master process listens to the network on port 9090 then?

  15. x 7 Silver badge

    why would you want a process to run while you're not logged in? Surely you need a user-initiated process to be tied to a user session, simply so you can control it - or terminate it. This complaint sounds illogical

    1. ckdizz

      It is illogical. It's an issue for people who hated that Debian moved to systemd. It's actually a default behaviour for systemd (it's discussed in the bug report on Debian), but seeing as some people don't see why they couldn't carry on using SysV, any excuse to point out flaws in systemd is used. The reality is that if you're expecting processes to run after you log out, you're not using systemd right - and indeed, you're not using your *nix system right at all. You've got used to lazy, insecure ways of working that SysV tolerates but systemd won't. If you want a process to carry on running after you log out, you have systemd look after it as a daemon.

      1. DainB Bronze badge

        "The reality is that if you're expecting processes to run after you log out, you're not using systemd right - and indeed, you're not using your *nix system right at all."

        Are you insane, trolling or systemd developer ?

        1. gypsythief

          They are none of those...

          ...but rather are drawing a distinction between processes and daemons:

          "The reality is that if you're expecting processes to run after you log out, you're not using systemd right - and indeed, you're not using your *nix system right at all... ...If you want a process to carry on running after you log out, you have systemd look after it as a daemon"

          In other words, processes, which are initated by the *user*, should be stopped when the *user* logs out.

          If you want the process to keep running, it should be run as a daemon where it has reduced privileges, can be managed by the *system*, and can therefore still be managed when the *user* has logged out.

          This is an important distinction that has been missed in both the article and the discussion that it seems systemd is only killing user processes on log out, not system deamons. This is how I would expect any OS to work, and indeed even Windows will do this.

          I am not a fan of systemd; I dislike anything that unneccessarily obfusctaes itself behind binary blobs (grub2, I'm looking at you...) but if one looks at this rationally instead of emotionally (maybe a difficult thing where systemd is concerned) then this is really expected and sensible behaviour.

          1. zanshin

            Re: They are none of those...

            I find myself wondering how such wildly divergent sets of experiences and expectations with the same OSes develop.

            In the world I have worked in for 20+ years, there's been no such distinction as the one being made here between daemons and processes . A "daemon" is loosely used to describe any process that doesn't end when its controlling terminal ends. In my experience, it's not a term reserved for init-managed processes.

            In my world, only daemons expected to start with the OS are put in the init system. Not all long-running, daemonic applications are allowed to start just because the OS started, as this could cause serious problems depending on the application and server in question. For direct start/stop control, these applications have control scripts that nohup or other-wise double-fork them into the background specifically so they will not end with the controlling terminal. They aren't run as root or as a "plain" user, but as an application-specific, generic account, usually using sudo or something similar. If you don't disconnect the process from the terminal of someone who started it, it will die, so every control script or program does this.

            I work in IT for an extremely large multinational, and to my knowledge no one there manages long-running Unix / Linux applications any other way than I describe above. If they start with the server, they go in an init config. Otherwise, they are, at some level, controlled by programs doing the equivalent of nohup specifically so they can be started "by hand". This is considered completely normal and not in any way onerous or dangerous, assuming privilege to run the control scripts is managed correctly (which is itself not very onerous, though not everyone does it as they should).

            Separately, and on my specific team, we have scheduled jobs that sometimes fail in ways that result in a need to re-run them manually. They are not long-running daemons in the sense of a web or database server, but they do take 4-6 hours to run to completion. We're not going to stick batch jobs in the init config, and we're not going to stay logged in for that long babysitting the process. Even if we have to manually check on its success when it's done, we don't want it to terminate mid-stream because our shell timed out or VPN dropped. We nohup the job and expect it to stay running.

            I don't know why anyone would think it's hard to control processes started this way. All you need is a PID and maybe sudo access to a control script that can use it. Except for some edge cases, it's pretty easy to either store the PID when you launch the process, or you can go find it after the fact with 'ps'. Sure, an init system that tracks the PID is cleaner and takes that bookkeeping off your hands, and would probably deal with those edge cases, but is that really considered that valuable in general?

            So the idea that this systemd behavior would be normal and expected for a Unix-like system is just incredibly strange to me. It's normal for Windows, not Unix/Linux. Based on the IT world I've been in, it feels like a solution to a problem no one ever mentioned.

    2. g00se
      Linux

      Why user processes should persist after logout?

      There are probably innumerable reasons. My own use case: play an audio file and close the laptop machine down at a time of my choice. In the meantime, i do NOT want or need to be logged in.

      I don't expect some asshat to ignore my nohup and at commands and treat my like some moron

      1. jch

        Re: Why user processes should persist after logout?

        I do this regularly. I kick off a long-running compile, for example, then I log out because I'm going home and I'm not going to be logged in.

        People have worked like this for a long time and now systemd comes along and says, no you can't do that, you must stay at work until 10pm watching your long running build run.

        What struck me as especially stupid was the comment that perhaps system users should be exempt from that policy. What's a system user? The user created for that application software you just installed? You're not retrospectively insisting that application software should have its user's uid < 1000 but those uids are informally reserved for system use, not application use.

        systemd needs a dose of real-life -- forcing your own desktop world view on everyone is preternaturally arrogant and stupid.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019