1. 19
    systemd sucks. linux rant suckless.org
  1.  

  2. 7

    I imagine most people reading this saw it already, but there was a thread here two months ago discussing a much more interesting essay about systemd. The older piece is also by someone disliking systemd, but it’s an existence proof that there are things to say that move the conversation forward, and that it can be relatively polite. Unlike this present article. :)

    The style of linking to recent systemd changelog entries and giving a one-sentence, out-of-context insult about each one… is probably intended to help sustain the fury of people who have already decided, which is certainly a valid political strategy. It’s pretty difficult to imagine that it would be useful to anyone trying to form an opinion.

    Also, clicking through most of those links, the titles they’re linked with are deeply deceptive. “CGI for systemd - the web thing has been discussed before.” That’s the entire comment. Yes, certainly it would be a less-than-great idea to run a web server within pid 1, or as a fundamental part of an init system. Neither is the case. The changelog entry that link goes to:

    When systemd forks off a new per-connection service instance it will now set the $REMOTE_ADDR environment variable to the remote IP address, and $REMOTE_PORT environment variable to the remote IP port. This behaviour is similar to the corresponding environment variables defined by CGI.

    This is talking about a feature inspired by inetd, whereby systemd listens on a network socket and starts up a short-lived instance of a service for each incoming connection. This is for raw services, not web scripts. The information it’s passing to the subprocess would be available by a call to getpeername(), so I certainly do find it strange and questionable that systemd has added this feature; I’m not sure who it’s meant to benefit.

    It’s a valid example of a quirky thing which, at best, will result in people writing things to integrate with it specifically, creating a lock-in effect with systemd. I’d need to know more about the intended audience to be firmly against it, but my first inclination is to dislike it.

    But notice that it’s completely unrelated to what the comment described it as. I’m sure that author, if asked, would express some rationale about how they expect people to know what they mean, but there’s no arguing that it strongly suggests a feature much easier to dislike than what this actually is. Saying this “has been discussed before” suggests the author thinks everyone should have read everything they’ve ever said, but the incoherent manner of presenting complaints in no particular grouping and with no mention of any perceived themes is a complete failure to communicate.

    The rules of persuasive writing say that I should have a total of three examples, to make my case without boring anyone. :) Here’s another item from the laundry list: “Calendar - As you see, your pid 1 should handle your calendars and cron jobs too.” The text it links to:

    Calendar time specifications in .timer units now also understand the strings “semi-annually”, “quarterly” and “minutely” as shortcuts (in addition to the preexisting “anually”, “hourly”, …).

    This was actually one of the flagship features of systemd, that it integrates the dependency-based invocation of traditional init scripts with the other major ways that background processes get invoked, citing inetd and cron as specific examples. By all means criticize that if you want, though personally I feel it’s one of the few clearly-good goals. But don’t bring it up again, a couple years into this debate, as if it’s a brand-new revelation. There’s no way to implement anything like cron jobs without handling calendars, and actually that’s not even the new part the change is introducing.

    This one’s not deceptive, nor is it nearly as inflammatory as the “CGI” thing, but I can’t actually imagine any way to defend the lack of context. That doesn’t move any conversation forward; it drags it on as long as possible, by encouraging everyone to memorize some talking points, and never question them. There is an unfavorable comparison to be made to public policy “debate”. Do we really want to make technical decisions that way? It isn’t inevitable; we all have a choice whether to engage or not with this sort of thing.

    One last example. From near the top, “screen brightness - Screen brightness is something that should crash your boot up when it is not working.” Linked text:

    When restoring the screen brightness at boot, stay away from the darkest setting or from the lowest 5% of the available range, depending on which is the larger value of both. This should effectively protect the user from rebooting into a black screen, should the brightness have been set to minimum by accident.

    I don’t see how this would cause a crash, as long as it handles errors properly if there’s no monitor plugged in. The author is certainly hinting that it might someday fail to, and reducing fragility is important, but this is a couple of system calls, nothing more.

    I spent a couple months this year fixing two co-occurring problems with a newly-installed NixOS machine, each of which independently resulted in a failed boot with nothing visible on screen (I did, eventually, make it work). It was immensely tedious trying to understand what was going on while having to find non-visual means of getting feedback from a system where neither userspace nor the kernel’s debug stuff ever came up. Removing a possible cause of booting to a black screen sounds like an excellent idea to me.

    I suppose I’m saying that screen brightness is already a cause of failure, whether systemd chooses to address or ignore it. I see the complaint, but…

    My main complaint against this author is a failure to sum up, or to draw any themes that they perceive. Putting out something that expresses anger, but not any coherent position that a reader could meaningfully agree or disagree with, is an act of anti-communication: It makes it more difficult for everyone who sees it to discuss the topic with each other.

    Secondarily, before I lost patience with these individual complaints, I was finding them all drastically oversimplified. By way of excusing this, the author says, “The list is exceeding the life-time of a philosopher to discuss all implications on the sanity of a system design.” Yes - perhaps don’t spew it at us, then?

    1. -11

      Shut up Donny!

      No, seriously, systemd is cancer. not directly because it’s too complex, not directly because it’s insecure but because it does not allow anything beside it. Freedom of choice is important, and I don’t want to live in a world where the major distributions become less and less different. It’s the Gnome monoculture all over again!

      And don’t get me started on Lennart bloody Poettering, who already deeply harmed the Linux ecosystem with Pulseaudio. And no, it is not good software, just admit it already. sndio shows how it should be done. The only reason why it was successful is because the fascists at RedHat backed it to support their plans for world domination.

      World domination I say? Yeah! Most people working on the beasts systemd and pulseaudio are RedHat developers. RedHat is with the NSA, as it is forced by law being a US-corporation. I bet my server rack the code is riddled with cryptic backdoors not easy to find. If placed there voluntarily or not doesn’t really matter though. These whales systemd and pulseaudio are will come crashing on us someday, because we haven’t learnt from the past.

      Just like Webkit, one has to wonder how valuable a program or ecosystem is which relies on big corporations to maintain it (except kernels). And the only half-secure way to run Webkit is to wrap it in 2 sandboxes inside a virtual machine. Do we really need this?

      Wake up, people!

      1. 3

        Shouldn’t that last be “wake up, sheeple!”?

        1. 3

          Except kernels? And why, pray tell, did you decide to single out kernels? What makes them so different?

          1. 2

            Who’s “Donny”?

            1. 3

              See The Big Lebowski.

        2. 14

          I don’t mind people bloviating about systemd, but I am sick like unto death of the rhetorical device of “the unix philosophy”. This is an ex post facto rationalization of a certain way of composing programs that a) has never been the organizing principle of any Unix since it escaped containment at AT&T and b) is an interesting ideology, but hardly one that has proved itself to be particularly useful in providing high-quality software.

          Like most religious ideologies, it’s largely used as a cudgel in stupid flamewars.

          1. 11

            Keep in mind that we live in a world in which it takes millions of lines of code just to layout some text in a Word document. We have this crazy large monolithic components that no one person knows what the hell is going on in them anymore. There was recently a bind vulnerability that could practically take down the internet, in code that the root nameservers weren’t even using. Software complexity leads to problems. We’ve seen it time and time and time and time again.

            See Some thoughts on security after 10 years of qmail

            1. 7

              Well, yes. Separation of concerns and well-defined interfaces and all that. Of course, if that’s the Unix philosophy, not only is it a sharp redefinition of the terms, it also simply doesn’t apply to the grotesque zoo of bug-ridden multiuse tools wading through a sludge of unstructured 7-bit text.

              I don’t disagree that those principles are important; I do strenuously disagree that they are the guiding principles of Unix development.

              1. 4

                well, you’d be wrong. Separation of concerns and well-defined interfaces happen to be the exact reason why Unix has dominated the computing world. Unix doesn’t get those things perfect, but it does way better than any other alternative that has ever existed.

                1. 2

                  Unix has only “dominated” the computing world since 2007; and I’d say that dominance has a lot more to do with path dependence and easy licensing than any technical reasons.

                  1. 2

                    Unix has only “dominated” the computing world since 2007

                    I don’t think I agree. Can you elaborate why you think Unix wasn’t dominant before?

                    1. 1

                      The overwhelming majority of computers that humans interacted with before iPhone were running Windows. The semantics of computing were defined by Microsoft. In fact, 2007 is generous; it’s really only been in the last five years that Android has overtaken Windows in any meaningful sense.

                      1. 3

                        The overwhelming majority of computers that humans interacted with before iPhone were running Windows.

                        That’s a bit workstation- or end-user-specific, which is only one corner of the computing world. Depending on your definitions, you might even say that Nokia was dominating the user-visible computing world at that time. And many other areas of computing were dominated by Unix. I see what you mean though.

                        The semantics of computing were defined by Microsoft.

                        Whose products were influenced by Unix, among others, quite greatly. Even MS-DOS, however primitive, was almost quoting Unix in places (did you know it had poor man’s version of pipes, with the same “|” syntax?). Windows NT had (has?) POSIX interface. And don’t forget the Internet, which really took Microsoft by surprise (and caused Windows to change, naturally).

                        In general, I think that generations of CS students growing on Unix had a lot to do with defining the semantic of computing.

                        In fact, 2007 is generous; it’s really only been in the last five years that Android has overtaken Windows in any meaningful sense.

                        And Android is not really the “Unix” of the “Unix philosophy” we’re talking about. Even MacOS X isn’t.

                        1. 2

                          But you can’t say that “the Unix philosophy” is determinative of Unix’s success and then play no true Scotsmen rhetorical games and handwave away the only Unixes that 99.9% of people will interact with because it’s obvious on the face of it that pipes ‘n ASCII has nothing at all to do with their success.

                          Look, I’ve been using Unix professionally for almost 25 years, and have lived on it in one form or another for almost that entire time – I’m not immune to its charms. I simply don’t accept the idea of a single guiding Unix philosophy as anything other than a rhetorical cudgel.

                          1. 3

                            But you can’t say that “the Unix philosophy” is determinative of Unix’s success

                            Oh, no, Unix has spread for various reasons.

                            and then play no true Scotsmen rhetorical games and handwave away the only Unixes that 99.9% of people will interact with because it’s obvious on the face of it that pipes ‘n ASCII has nothing at all to do with their success.

                            And I’m not playing them. As for me, the part that makes Unix Unix is the organiing principle of the system and some parts most end-users seldom interact with, even on Unix-based systems.

                            I actually never saw the point of pushing Unix to the desktop as hard as the Linux crowd has been doing for the last twenty years. It’s just not that good at desktoping. I use X11 exclusively on the desktop myself, but I wouldn’t want my daughter to marry one.

                            I simply don’t accept the idea of a single guiding Unix philosophy as anything other than a rhetorical cudgel.

                            But not having a rigorous definition doesn’t make the concept meaningless. At the risk of sounding like a hipster, Unix is somewhat like a culture. There’s no supreme leader or founding document, but some ideas and attitudes are more prevalent than others. So when people perceive systemd as breaking too many (possibly unspoken) conventions, it’s natural to react with “we don' like these sort of things ‘round here”, and Poettering responds in kind — I remember him saying something like “here’s another person who likes Unix”.

                            I guess what I’m trying to say is, of course there’s no single guiding principle, but that doesn’t make the argument a complete bullshit.

                            1. 1

                              I guess what I’m trying to say is, of course there’s no single guiding principle, but that doesn’t make the argument a complete bullshit.

                              This is totally reasonable. I’m just easily annoyed, I guess.

                            2. 1

                              But you can’t say that “the Unix philosophy” is determinative of Unix’s success

                              When people talk about the Unix Philosophy I don’t think they are talking about it being a reason for its success, just that it’s a good design principle.

                  2. 2

                    I merely pointed out that software is too complex and needs to be simpler, not that the unix philosophy is some golden set of rules to live by. I may believe that, but my comment, hopefully, avoided the associated dogma. The Unix Philosophy (depending on who you quote, of course), however, speaks to simplicity and composition, notions which, as DJB points out in the qmail paper, leads to secure, useful software.

                    emphasis on my edits

                    1. 3

                      I think we’re probably very close to believing the same thing, actually. I wasn’t trying to call you out or anything; if I did so, that was an error on my part, for which I apologize. No need to bring more conflict into the world.

                2. [Comment removed by author]

                  1. 9

                    Funny enough, composable problems and “map reduce” could also be described as the lisp philosophy, which is otherwise practically the antithesis of Unix philosophy. If there’s a compelling reason to do something, that reason probably has its own name, and we can avoid arguments about “true Unix” entirely.

                    1. 3

                      Exactly this. “The Unix Philosophy” always always always descends into true Scotsman territory. It’s a conceit best abandoned.

                3. 5

                  What I don’t get is why anyone wants systemd on servers. I can understand the argument for it on desktops (although I don’t agree), but all of the benefits I’ve heard of systemd don’t seem to make any sense on a server. Faster bootup times and bootup profiling? I boot servers so rarely I don’t really care. All of the logging things? On most servers I’m pushing logs to something like Splunk or an aggregation service anyways. Plane logs are quite nice and simple. In the age of microservices I’m also generally only running one thing per server so a more expressive language and dependencies between them isn’t really that helpful. I happen to think that if something like daemontools doesn’t solve your problem on a server you’re probably doing something wrong.

                  1. 3

                    Honestly, the declarative service files are a lifesaver. Know what happens when you ask a java developer for an init script? You get 300 lines of bash full of hardcoded paths and complete disregard for starting things in the real world. Hell, even a sysadmin will have trouble with an initscript for the first handful of times. Copying a .service off of the arch linux wiki and changing the StartExec line will work 90% of the time, whereas doing the same with an initscript will result in it being subtly broken every time.

                    That said, if something like runsv were widely used, that’s practically a utopia.

                    1. 2

                      Haha, I’ve got good money that says that Java developer will just have the systemd service run a nasty shell script. Or maybe it’ll just call their init script.

                      But you don’t need systemd to have declarative start scripts, seems a bit like keeping the bath water to keep the baby. I’m a daemontools advocate and even though daemontool run scripts tend to be in shell I haven’t found it problematic, mostly because there is very little interesting things you can be doing in a run script.

                    2. 0

                      There are numerous reasons why would want systemd on servers. On multi-user servers, user sessions are incredibly useful, for example. Socket activation is incredible when you’re upgrading your daemons. Resource management becomes easier too, and there are numerous other benefits.

                      But this is going off-topic here, to be honest.

                      1. 5

                        Honestly. Utter balderdash. Socket activation has nothing to do with upgrading your daemons; the concept of users existed well before systemd; and please be precise about what ‘resource management’ gets ‘easier’.

                        1. -1

                          Users != user sessions, and socket activation (the part of it where systemd opens the socket for you, and keeps it open too) does help with daemon upgrades, among other things.

                          There are a zillion resources on the internet that explain the benefits, please use your favourite search engine. I’m not going to try and summarize it in a comment.

                        2. 3

                          Would you say those are the strongest arguments or just some of them? What you have listed doens’t jive with my experiences on modern servers. For the most part, the push is to make user logins the exception. It’s unclear to me what you mean by socket activation being incredible for upgrades. For the most part taht can be handled by load balancers and/or the semantics of the system expecting failed requests. And I’m not sure what you mean by resource management, specially in the cloud/container world where every thing does one thing and dies when done.

                          Maybe you have good arguments, but your response had no useful details and was just waving your hands around saying “no it’s great for all these things”. In contrast, my post had specific situations and why I felt they were not benefited by something like systemd.

                      2. 10

                        Yet another rant that mixes systemd as pid1 with the systemd set of tools. While a lot of the complaints the article makes may be valid, this conflation sets such an annoying, ranty tone, that one just shakes his head and walks away without reading ‘till the end.

                        Shame.

                        1. 12

                          I disagree. The systemd developers are the ones who bundle a ton of tools with their unit system, and as such, it’s valid to criticize both the tools and the init system when criticizing systemd.

                          1. 5

                            That’s a valid criticism. Conflating that with “everything is in PID1!!!!” is not. I do not know what you disagree with in my statement, because there’s nothing in it that is in disagreement with what you wrote.

                            1. 8

                              There is a significant interdependency between the various systemd components that you cannot ignore.

                              1. 1

                                Yes. Still doesn’t make them PID1, and that’s a huge difference. Those fond of the UNIX philosophy should be especially aware of that.

                                1. 1

                                  I have heard multiple people say that it’s highly modular. My interpretation of what they have said is that you can replace the binaries with other ones, but not that the functionality is not required? Do you think that interpretation is accurate? If so, to me the problem seems to be not if systemd is modular or not but rather that there is too much there, regardless of if you can swap out implementations.

                            2. 3

                              I wish the first response to every criticism of systemd wasn’t an irrelevant handwave about systemd not being just pid1.

                            3. 6

                              I think it should be highlighted that suckless is writing their own init system. However, when someone is writing a “competing” tool, it doesn’t inspire much confidence in the argument (or the software, really) when their criticism of the other tool starts off like this:

                              There is a menace which is spreading like a disease throughout the Linux world, it is called systemd.

                              Think you can write a better init system? Great. Just ship your code, and enforce its superiority through correct design decisions. Let me know when a bunch of distributions have decided sinit is better than systemd (or even SysVInit, for that matter).

                              1. 11

                                While it’s not an excuse for inflammatory language, keep in mind that this tired debate is basically hijacked by very loud, very obnoxious voices. So it’s understandable that some people who dislike systemd might get angry at constant accusations of extreme conservatism and incompetence. Angry enough to not being able to present their criticism without that anger showing. Same goes for pro-systemd folks.

                                Also:

                                I think it should be highlighted that suckless is writing their own init system.

                                uwotm8. A dumb init isn’t an init system, even by very lax definitions of such a thing. sinit is an init, it’s dumb as a brick and in no way is it enough to boot a system to a useable state.

                                It’s not completely honest, btw, to invoke the pure technical merit argument. Software rarely rises on pure technical merit, there’s a lot of politics (not talking evil conspiracies here) in the opensource community.

                                1. 7

                                  sinit, like most other suckless projects, does a single thing well, simply, and rests on the laurels of the Unix philosophy.

                                  1. 2

                                    I’m not criticizing sinit here. Just being anal about widely used (even relatively ambigious) terms.

                                    1. 3

                                      I know. I was just making the simplest argument, without inflammatory language, for sinit.

                                      I’m not at all sure why more arguments systemd v. any other init system don’t just boil down to the statement I just made. But, they don’t; Because people are terrible.

                              2. 3

                                He didn’t do as much homework as he should have. systemd was in no way inspired by MS Windows that I’m aware of. I’ve seen several remarks about OS X’s init.d by Lennart Poettering but I’ve never heard him say or seen him write anything about Windows being an inspiration.

                                Perhaps the Windows comment was just to draw on the ire of open source folks stereotypically have against the OS. A little psychology.

                                1. 4

                                  Honestly, if anyone spent any time looking at the code instead of writing vitriolic comments on the internet, they’d realize that systemd is basically launchd for Linux - no, not the half-assed launchd port that people have tried to make, but a real, Linux software stack integrated version. It uses dbus to communicate with daemons instead of mach ports, since most of the system daemons it wants to talk to use dbus. It uses the same “don’t daemonize, just run from main()” principle. It has service configuration files that are plain and simple keyfiles similar to Apple’s plists. It even has a suite of replacement daemons for a very similar set of replacement daemons that Apple replaced when they switched to launchd.

                                  Most people have nothing bad to say about launchd. It does its job significantly better than the SysV/SystemStart shit it replaced in Tiger. It’s fast. It’s unobtrusive. Most people literally don’t even know of its existence. Meanwhile systemd gets called “literally Hitler,” with most of the hate of systemd coming directly from the fact that they literally have no clue what systemd is or does, just that the internet told them to hate it, and they hate change. They hear things like “systemd replaces cron,” and they immediately think “wow, that thing’s one horrendously complicated PID1,” not “okay so systemd now includes a crond implementation in its source repo that is designed to work with systemd, exactly as launchd does.”

                                  Yeah, it sucks that Lennart et. al. are making everyone have to get along by simplifying their daemons, making configuration incredibly simple albeit maybe not through a format they prefer, etc. Linux has rampantly been about freedom since very early on, and they have the instinctual fear of anything that limits their freedom - systemd appears to limit that freedom. But let’s grow up people: systems are complicated.

                                  The number of services a typical Linux system has is growing very quickly, and the number of interactions between those systems are skyrocketing. Handwriting scripts is just too error prone of a process. They can’t properly account for the conditions the system can get into (we’ve tried for literally decades at this point and we still end up with broken systems), and worse, init scripts are code that attempts to encode policy instead of policy enforced by code - that’s a cart-in-front-of-horse problem you literally cannot solve; you’re building a system where the only way to understand the policy is to understand exactly every possible operating state of the code, instead of simply encoding the state you want the system to be in and having the init system solve for it.

                                  We need smarter software, period. No matter how many anti-systemd trolls fud up the thread, they can’t change the fact that systemd has essentially solved this problem, which is why it’s so universally adopted. It’s over. Let’s move on already. I hear containers are still interesting to fight about, who’s up for a round of Docker vs Rkt articles?

                                  1. 5

                                    It uses dbus to communicate with daemons instead of mach ports, since most of the system daemons it wants to talk to use dbus.

                                    Which they do because the crowd roughly associated with systemd either wrote these daemons or forced DBus down the authors' throats.

                                    Meanwhile systemd gets called “literally Hitler,”

                                    Is it? Funny.

                                    with most of the hate of systemd coming directly from the fact that they literally have no clue what systemd is or does,

                                    But how can we? It does more and more every week.

                                    and they hate change

                                    I, for one, hate change for no reason. I feel that every couple of years yet another incompatible Linux sound system is forced upon me, before the previous one has had any chance to achieve stability, and things don’t become better as the result, it’s just that the existing set of flaws is replaced by a new set of flaws. Now, if these changes were solving real problems, fine. And some of them do, to be fair: configuring network interfaces as a user is a need that profileration of laptops brought, and network manager solves it, however poorly. But in most cases I just see change for the sake of being all new, cool and cutting edge. See also: CADT.

                                    by simplifying their daemons

                                    Are you kidding me? Whatever one may say about systemd, it’s not simple.

                                    But let’s grow up people: systems are complicated.

                                    Therefore they should be modular, without opaque highly coupled systems that do everything.

                                    The number of services a typical Linux system has is growing very quickly, and the number of interactions between those systems are skyrocketing.

                                    Maybe we need less interaction and interdependency.

                                    Handwriting scripts is just too error prone of a process.

                                    This is true. On the other hand, it gives you flexibility and the ability to do the same operations from an interactive shell. But SysV-style scripts and systemd are not the only approaches, you know.

                                    you’re building a system where the only way to understand the policy is to understand exactly every possible operating state of the code

                                    So does Poettering, but without shell scripts.

                                    We need smarter software, period.

                                    You saying “period” doesn’t make it so. And smarter is not the same as more complex, often even the opposite.

                                    No matter how many anti-systemd trolls fud up the thread

                                    Thanks.

                                    Edited to add: I understand that one of the complaints against Poettering and the systemd crowd was precisely this attitude: dismissing users' concerns, complaints and dissatisfaction as irrelevant at best and as trolling at worst, coupled with the “we know best what’s good for you, silly users, and for you, silly authors of other silly system services” approach. Look who’s trolling now.

                                    systemd has essentially solved this problem, which is why it’s so universally adopted

                                    The problem has been solved before, systemd solved it badly, and it was adopted for wholly different reasons. You’re not going to claim that anything popular is so because it’s good, are you?

                                    1. 2

                                      I wouldn’t run OS X as a server either, though. As I said in another post, I can possibly buy systemd for desktop, but for a server I see very little benefit. I want simplicity.

                                      But let’s grow up people: systems are complicated.

                                      So let’s make them simpler.