1. 34

    Good talk.

    I recently used systemd “in anger” for the first time on a raspi device to orchestrate several scripts and services, and I was pleasantly surprised (but also not surprised, because the FUD crowd is becoming more and more fingerprintable to me). systemd gives me lifecycle, logging, error handling, and structure, declaratively. It turns out structure and constraints are really useful, this is also why go has fast dependency resolution.

    It violates unix philosohpy

    That accusation was also made against neovim. The people muttering this stuff are slashdot markov chains, they don’t have any idea what they’re talking about.

    1. 22

      The declarative units are definitely a plus. No question.

      I was anti-systemd when it started gaining popularity, because of the approach (basically kitchen-sinking a lot of *NIX stuff into a single project) and the way the project leader(s) respond to criticism.

      I’ve used it since it was default in Debian, and the technical benefits are very measurable.

      That doesnt mean the complaints against it are irrelevant though - it does break the Unix philosophy I think most people are referring to:

      Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”.

      1. 30

        If you believe composability (one program’s output is another program’s input) is an important part of The Unix Philosophy, then ls violates it all day long, always has, likely always will. ls also violates it by providing multiple ways to sort its output, when sort is right there, already doing that job. Arguably, ls formatting its output is a violation of Do One Thing, because awk and printf exist, all ready to turn neat columns into human-friendly text. My point is, The Unix Philosophy isn’t set in stone, and never has been.

        1. 7

          Didn’t ls predate the Unix Philosophy? There’s a lot of crufthistory in unix. dd is another example.

          None of that invalidates the philosophy that arose through an extended design exploration and process.

          1. 4

            nobody said it’s set in stone; it’s a set of principles to be applied based on practicality. like any design principle, it can be applied beyond usefulness. some remarks:

            • i don’t see where ls violates composability. the -l format was specifically designed to be easy to grep.
            • the sorting options are an example of practicality. they don’t require a lot of code, and would be much more clumsy to implement as a script (specifically when you don’t output the fields you’re sorting on)
            • about formatting, i assume you’re referring to columniation, which to my knowledge was not in any version of ls released by Bell Labs. checking whether stdout is a terminal is indeed an ugly violation.
            1. 6

              i don’t see where ls violates composability. the -l format was specifically designed to be easy to grep.

              People have written web pages on why parsing the output of ls is a bad idea. Using ls -l doesn’t solve any of these problems.

              As a matter of fact, the coreutils people have this to say about parsing the output of ls:

              However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited.

              Moving on…

              the sorting options are an example of practicality. they don’t require a lot of code, and would be much more clumsy to implement as a script (specifically when you don’t output the fields you’re sorting on)

              This cuts closer to the point of what we’re saying, but here I also have to defend my half-baked design for a True Unix-y ls Program: It would always output all the data, one line per file, with filenames quoted and otherwise prepared such that they always stick to one column of one line, with things like tab characters replaced by \t and newline characters replaced by \n and so on. Therefore, the formatting and sorting programs always have all the information.

              But, as I said, always piping the output of my ls into some other script would be clumsier; it would ultimately result in some “human-friendly ls” which has multiple possible pipelines prepared for you, selectable with command-line options, so the end result looks a lot like modern ls.

              about formatting, i assume you’re referring to columniation, which to my knowledge was not in any version of ls released by Bell Labs. checking whether stdout is a terminal is indeed an ugly violation.

              I agree that ls shouldn’t check for a tty, but I’m not entirely convinced no program should.

              1. 4

                just because some people discourage composing ls with other programs doesn’t mean it’s not the unix way. some people value the unix philosophy and some don’t, and it’s not surprising that those who write GNU software and maintain wikis for GNU software are in the latter camp.

                your proposal for a decomposed ls sounds more unixy in some ways. but there are still practical reasons not to do it, such as performance and not cluttering the standard command lexicon with ls variants (plan 9 has ls and lc; maybe adding lt, lr, lu, etc. would be too many names just for listing files). it’s a subtle point in unix philosophy to know when departing from one principle is better for the overall simplicity of the system.

          2. 25

            With all due respect[1], did your own comment hit your fingerprint detector? Because it should. It’s extrapolating wildly from one personal anecdote[2], and insulting a broad category of people without showing any actual examples[3]. Calling people “markov chains” is fun in the instant you write it, but contributes to the general sludge of ad hominem dehumanization. All your upvoters should be ashamed.

            [1] SystemD arouses strong passions, and I don’t want this thread to devolve. I’m pointing out that you’re starting it off on the wrong foot. But I’m done here and won’t be responding to any more name-calling.

            [2] Because God knows, there’s tons of badly designed software out there that has given people great experiences in the short term. Design usually matters in the long term. Using something for the first time is unlikely to tell you anything beyond that somebody peephole-optimized the UX. UX is certainly important, rare and useful in its own right. But it’s a distinct activity.

            [3] I’d particularly appreciate a link to NeoVim criticism for being anti-Unix. Were they similarly criticizing Vim?

            1. 9

              [3] I’d particularly appreciate a link to NeoVim criticism for being anti-Unix. Were they similarly criticizing Vim?

              Yes, when VIM incorporated a terminal. Which is explicitly against its design goals. From the VIM 7.4 :help design-not

              VIM IS... NOT                                           *design-not*
              
              - Vim is not a shell or an Operating System.  You will not be able to run a
                shell inside Vim or use it to control a debugger.  This should work the
                other way around: Use Vim as a component from a shell or in an IDE.
                A satirical way to say this: "Unlike Emacs, Vim does not attempt to include
                everything but the kitchen sink, but some people say that you can clean one
                with it.  ;-)"
              

              Neo-VIM appears to acknowledge their departure from VIM’s initial design as their :help design-not has been trimmed and only reads:

              NVIM IS... NOT                                          design-not
              
              Nvim is not an operating system; instead it should be composed with other
              tools or hosted as a component. Marvim once said: "Unlike Emacs, Nvim does not
              include the kitchen sink... but it's good for plumbing."
              

              Now as a primarily Emacs user I see nothing wrong with not following the UNIX philosophy, but at it is clear that NeoVIM has pushed away from that direction. And because that direction was an against their initial design it is reasonable for users that liked the initial design to criticism NeoVIM because moving further away from the UNIX philosophy.

              Not that VIM hadn’t already become something more than ‘just edit text’, take quickfix for example. A better example of how an editor can solve the same problem by adhering to the Unix Philosophy of composition through text processing would be Acme. Check out Acme’s alternative to quickfix https://youtu.be/dP1xVpMPn8M?t=551

              1. 0

                akkartik, which part of my comment did you identify with? :) FWIW, I’m fond of you personally.

                I’d particularly appreciate a link to NeoVim criticism for being anti-Unix

                Every single Hacker News thread about Neovim.

                Were they similarly criticizing Vim?

                Not until I reply as such–and the response is hem-and-haw.

                1. 9

                  To be fair I don’t think the hacker news hive mind is a good judge of anything besides what is currently flavour of the week.

                  Just yesterday I had a comment not just downvoted but flagged and hidden-by-default, because I suggested Electron is a worse option than a web app.

                  HN is basically twitter on Opposite Day: far too happy to remove any idea even vaguely outside what the group considers “acceptable”.

                  1. 4

                    Indeed, I appreciate your comments as well in general. I wasn’t personally insulted, FWIW. But this is precisely the sort of thing I’m talking about, the assumption that someone pushing back must have their identity wrapped up in the subject. Does our community a disservice.

                    1. 0

                      OTOH, I spent way too much of my life taking the FUD seriously. The mantra-parroting drive-by comments that are common in much of the anti-systemd and anti-foo threads should be pushed back. Not given a thoughtful audience.

                      1. 2

                        Totally fair. Can you point at any examples?

                        1. 3

                          https://news.ycombinator.com/item?id=7289935

                          The old Unix ways are dying… … Vim is, in the spirit of Unix, a single purpose tool: it edits text.

                          https://news.ycombinator.com/item?id=10412860

                          thinks that anything that is too old clearly has some damage and its no longer good technology, like the neovim crowd

                          Also just search for “vim unix philosophy” you’ll invariably find tons of imaginary nonsense:

                          https://hn.algolia.com/?query=vim%20unix%20philosophy&sort=byPopularity&prefix&page=0&dateRange=all&type=comment

                          Please don’t make me search /r/vim :D

                          1. 4

                            thinks that anything that is too old clearly has some damage and its no longer good technology, like the neovim crowd

                            That’s not saying that neovim is ‘anti-Unix philosophy’, it’s saying that neovim is an example of a general pattern of people rewriting and redesigning old things that work perfectly well on the basis that there must be something wrong with anything that’s old.

                            Which is indeed a general pattern.

                            1. 1

                              That’s not saying that neovim is ‘anti-Unix philosophy’

                              It’s an example of (unfounded) fear, uncertainty, and doubt.

                              rewriting and redesigning old things that work perfectly well on the basis that there must be something wrong with anything that’s old.

                              That’s a problem that exists, but attaching it to project X out of habit, without justification, is the pattern I’m complaining about. In Neovim’s case it’s completely unfounded and doesn’t even make sense.

                              1. 1

                                It’s not unfounded. It’s pretty obvious that many of the people advocating neovim are doing so precisely because they think ‘new’ and ‘modern’ are things that precisely measure the quality of software. They’re the same people that change which Javascript framework they’re using every 6 weeks. They’re not a stereotype, they’re actual human beings that actually hold these views.

                                1. 2

                                  Partial rewrite is one of the fastest ways to hand off software maintainership, though. And vim needed broader maintainer / developer community.

                                  1. 0

                                    Vim’s maintainer/developer community is more than sufficient. It’s a highly extensible text editor. Virtually anything can be done with plugins. You don’t need core editor changes very often if at all, especially now that the async stuff is in there.

                                    1. 3

                                      You don’t need core editor changes very often if at all, especially now that the async stuff is in there.

                                      Which required pressure from NeoVim, if I understood the situation correctly. Vim is basically a one-man show.

                            2. 2

                              Thanks :) My attitude is to skip past crap drive-by comments as beneath notice (or linking). But I interpreted you to be saying FUD (about SystemD) that you ended up taking seriously? Any of those would be interesting to see if you happen to have them handy, but no worries if not.

                              Glad to have you back in the pro-Neovim (which is not necessarily anti-Vim) camp!

                  2. 20

                    What is FUD is this sort of comment: the classic combination of comparing systemd to the worst possible alternative instead of the best actual alternative with basically claiming everyone that disagrees with you is a ‘slashdot markov chain’ or similar idiotic crap.

                    On the first point, there are lots of alternatives to sysvinit that aren’t systemd. Lots and lots and lots. Some of them are crap, some are great. systemd doesn’t have a right to be compared only to what it replaced, but also all the other things that could have replaced sysvinit.

                    On the second point, it’s just bloody rude. But it also shows you don’t really understand what people are saying. ‘I think [xyz] violates the unix philosophy’ is not meaningless. People aren’t saying it for fun. They’re saying it because they think it’s true, and that it’s a bad thing. If you don’t have a good argument for the Unix philosophy not matter, or you think systemd doesn’t actually violate it, please go ahead and explain that. But I’ve never actually seen either of those arguments. The response to ‘it violates the Unix philosophy’ is always just ‘shut up slashdotter’. Same kind of comment you get when you say anything that goes against the proggit/hn hivemind that has now decided amongst other things that: microsoft is amazing, google is horrible, MIT-style licenses are perfect, GPL-style licenses are the devil-incarnate, statically typed languages are perfect, dynamically typed languages are evil, wayland is wonderful, x11 is terrible, etc.

                    1. 8

                      claiming everyone that disagrees with you is a ‘slashdot markov chain’ or similar idiotic crap

                      My claim is about the thoughtless shoveling of groundless rumors. Also I don’t think my quip was idiotic.

                      there are lots of alternatives to sysvinit that aren’t systemd

                      That’s fine, I never disparaged alternatives. I said: systemd is good and I’m annoyed that the grumblers said it wasn’t.

                      1. 2

                        It’s not good though, for all the reasons that have been said. ‘Better than what you had before’ and ‘good’ aren’t the same thing.

                        1. 1

                          seriously. If you don’t like systemd, use something else and promote its benefits. Tired of all the talking down of systemd. It made my life so much easier.

                          1. 1

                            seriously. If you like systemd, use it and shut up about it. Tired of all the talking up of systemd as if it’s actually any better than its alternatives, when it is objectively worse, and is poorly managed by nasty people.

                            1. 4

                              Have you watched the video this thread is about? Because you really sound like the kind of dogmatist the presenter is talking about.

                              If you like systemd, use it and shut up about it

                              Also, isn’t this a double-standard, since when it comes to complaining about systemd, this attitude doesn’t seem that prevalent.

                              1. 2

                                No, because no other tool threatens the ecosystem like systemd does.

                                Analogy: it wasn’t a double-standard 10 years ago to complain about Windows and say ‘if you like Windows, use it and shut up about it’.

                                1. 3

                                  I see this kind of vague criticism when it comes to systemd alot. What ecosystem is it really breaking? It’s all still open source, there aren’t any proprietary protocols or corporate patents that prevent people from modifying the software to not have to rely on systemd. This “threat”, thr way I see it, has turned out to be at most a “ minor inconvenience “.

                                  I suppose you’re thinking about examples like GNOME, but on the one hand, GNOME isn’t a unix-dogmatist project, but instead they aim to create a integrated desktop experience, consciously trading this in for ideal modularity – and on the other, projects like OpenBSD have managed to strip out what required systemd and have a working desktop environment. Most other examples, of which I know, have a similar pattern.

                    2. 6

                      I think that the problem is fanboyism, echo chambers and ideologies.

                      I might be wrong, so please don’t consider this an accusation. But you writing this sounds like someone hearing that systemd is bad, therefore never looking at it, yet copying it. Then one tries it and finding out that baseless prejudices were in fact baseless.

                      After that the assumption is that everyone else must have been doing the same and one is enlightened now to see it’s actually really cool.

                      I think that this group behavior and blindly copying opinions is one of the worst things in IT these days, even though of course it’s not limited to this field.

                      A lot of people criticizing systemd actually looked at systemd, really deep, maybe even built stuff on it, or at least worked with it in production as sysadmin/devop/sre/…

                      Yes, I have used systemd, yes I understand why decisions we’re taken, where authors if the software were going, read specs of the various parts (journald for example), etc.

                      I think I have a pretty good understanding compared to at least most people that only saw it from a users perspective (considering writing unit files to be users perspective as well).

                      So I could write about that in my CV and be happy that I can answer a lot of questions regarding systemd, advocate its usage to create more demand and be happy.

                      To sum it up: I still consider systemd to be bad on multiple layers, both implementation and some ideas that I considered great but then through using it seeing that it was a wrong assumption. By the way that’s the thing I would not blame anyone for. It’s good that stuff gets tried, that’s how research works. It’s not the first and not the last project that will come out sounding good, to only find out a lot of things either doesn’t make a difference or make it worse.

                      I am a critic of systemd but I agree that there’s a lot of FUD as well. Especially when there’s people that blame everything, including own incompetence on systemd. Nobody should ever expect a new project to be a magic bullet. That’s just dumb and I would never blame systemd for trying a different approach or for not being perfect. However I think it has problems on many levels. While I think the implementation isn’t really good that’s something that can be fixed. However I think some parts of the concept level are either pretty bad or have turned out to be bad decisions.

                      I was very aware that especially in the beginning the implementation was bad. A lot got better. That’s to be expected. However next to various design decisions I consider bad I think many more were based on ideas that I think to most people in IT sound good and reasonable but in the specific scenarios that systemd is used it at least in my experience do not work out at all or only work well in very basic cases.

                      In other words the cases where other solutions are working maybe not optimal, but that aren’t considered a problem worth fixing because the added complexity isn’t worth it systemd really shines. However when something is more complex I think using systemd frequently turns out to be an even worse solution.

                      While I don’t wanna go into detail because I don’t think this is the right format for an actual analysis I think systemd in this field a lot in common with both configuration management and JavaScript frameworks. They tend to be amazing for use cases that are simple (todo applications for example), but together with various other complexities often make stuff unnecessarily complicated.

                      And just like with JavaScript frameworks and configuration management there’s a lot of FUD, ideologies, echochambers, following the opinion of some thought leaders, and very little building your own solid opinion.

                      Long story short. If you criticize something without knowing what it is about then yes that’s dumb and likely FUD. However assuming that’s the only possible reason for someone criticizing software is similarly dumb and often FUD regarding this opinion.

                      This by the way also works the reverse. I frequently see people liking software and echoing favorable statements for the same reasons. Not understanding what they say, just copying sentences of opinion leaders, etc.

                      It’s the same pattern, just the reversal, positive instead of negative.

                      The problem isn’t someone disliking or liking something, but that opinions and thoughts are repeated without understanding which makes it hard to have discussions and arguments that give both sides any valuable insides or learnings

                      Then things also get personal. People hate on Poetteing and think he is dumb and Poetteing thinks every critic is dumb. Just because that’s a lot of what you see when every statement is blindly echoed.

                      1. 1

                        That’s nice, but the implication of the anti-systemd chorus was that sys v init was good enough. Not all of these other “reasonable objections” that people are breathless to mention.

                        The timbre reminded me of people who say autotools is preferrable to cmake. People making a lot of noise about irrelevant details and ignoring the net gain.

                        But you writing this sounds like someone hearing that systemd is bad, therefore never looking at it, yet copying it.

                        No, I’m reacting to the idea that the systemd controversy took up any space in my mind at all. It’s good software. It doesn’t matter if X or Y is technically better, the popular narrative was that systemd is a negative thing, a net-loss.

                        1. 2

                          In your opinion it’s good software and you summed up the “anti-systemd camp” with “sys v init was good enough” even though people from said “anti-systemd camp” on this very thread disagreed that that was their point.

                          To give you an entirely different point of view, I’m surprised you don’t want to know anything about a key piece of a flagship server operating systems (taking that one distro is technically an OS) affecting the entire eco system and unrelated OS’ (BSDs etc.) that majorly affects administration and development on Linux-based systems. Especially when people have said there are clear technical reasons for disliking the major change and forced compliance with “the new way”.

                          1. 2

                            you summed up the “anti-systemd camp” with “sys v init was good enough” even though people from said “anti-systemd camp” on this very thread disagreed that that was their point.

                            Even in this very thread no one has actually named a preferred alternative. I suspect they don’t want to be dragged into a discussion of details :)

                            affecting the entire eco system and unrelated OS’ (BSDs etc.)

                            BSDs would be a great forum for demonstrating the alternatives to systemd.

                            1. 2

                              Well, considering how many features that suite of software has picked up, there isn’t currently one so that shortens the conversation :)

                              launchd is sort of a UNIX alternative too, but it’s currently running only on MacOS and it recently went closed source.

                      2. 3

                        It violates unix philosohpy

                        That accusation was also made against neovim. The people muttering this stuff are slashdot markov chains, they don’t have any idea what they’re talking about.

                        i don’t follow your reasoning. why is it relevant that people also think neovim violates the unix philosophy? are you saying that neovim conforms to the unix philosophy, and therefore people who say it doesn’t must not know what they’re talking about?

                        1. 1

                          are you saying that neovim conforms to the unix philosophy, and therefore people who say it doesn’t must not know what they’re talking about?

                          When the implication is that Vim better aligns with the unix philosophy, yes, anyone who avers that doesn’t know what they’re talking about. “Unix philosophy” was never a goal of Vim (”:help design-not” was strongly worded to that effect until last year, but it was never true anyways) and shows a deep lack of familiarity with Vim’s features.

                          Some people likewise speak of a mythical “Vim way” which again means basically nothing. But that’s a different topic.

                          1. 1

                            vim does have fewer features which can be handled by other tools though right? not that vim is particularly unixy, but we’re talking degrees

                        2. 1

                          The people muttering this stuff are slashdot markov chains, they don’t have any idea what they’re talking about

                          I’ll bookmark this comment just for this description.

                        1. 3

                          My brother’s startup (Think.Dev) is looking for a mobile developer for iOS to develop the initial frontend for a ~2 month contract, possibly more work afterwards too. It’s an event-based meetup app.

                          1. 1

                            I wanted to put way more information on the listing but it turns out it’s effectively just a single line it allows you to post, is this a limitation of LinkedIn or was this a choice for the board to have the actual listings somewhere else?

                            1. 1

                              Huh, that’s a really strange limitation and I don’t understand it. I’ve reviewed the settings pages and I can’t see anything like that. When I click “Start a conversation” or “Post a job” I’m prompted for a title and body, and the body text takes several hundred words of lorem ipsum without complaint. If you reload and still can’t post more than one line, please message your posting to me (here or on LinkedIn) and I’ll add it.

                              1. 1

                                It was me being silly in the end. It said “Add a link” in the body and I thought it actually was just for adding a link and tried to cram the whole thing into the title, yet another lesson about late night work output. Thanks for letting me sort it out without getting banned.

                                1. 1

                                  Unlike my other inboxes on LinkedIn, it has not turned into a steady stream of recruiter spam, so I feel pretty relaxed about the whole thing.

                            1. 1

                              Sent a request too, might be in the market soon!

                              1. 3

                                Does this apply to (a) businesses with customers resident in California, (b) businesses incorporated in California with customers anywhere, or (c) businesses incorporated in California with customers resident in California?

                                This seems tricky to do at the state level and should be federal…Similar to how GDPR was EU level.

                                1. 4

                                  California has a long history of setting a higher (per-state) bar on things like vehicle emissions and mileage, forcing companies to either build two of everything (“49-state” cars) or follow the stricter set of rules for everything – usually they decide to do the latter. California’s lawmakers are probably hoping this will go the same way, since GDPR led the way.

                                  1. 2

                                    It seems they’re doing it to get the ball rolling so to speak. Europe, now California, who next?

                                  1. 5

                                    Because knock packets use a timestamp to limit knock-reuse attacks, servers and clients must have synchronized clocks. Clock skew greater than 60 seconds is likely to cause the knock process to fail, resulting in a “connection refused” error when establishing the TCP connection.

                                    Given this, if my server’s clock does fall out of sync, how do I connect to it to fix it if Oxy is my only access method?

                                    1. 1

                                      Remember to set up ntp when you set up your server.

                                      1. 1

                                        Software crashes and networks disconnect. Even if you start NTP you can still end up with an inaccessible server.

                                        1. 0

                                          So if you have no network connectivity for ntp to update, how are you going to log in with any remote access tool?

                                          Software crash of ntpd is unrealistic.

                                          1. 1

                                            NTP has an exponential back off mechanism for retrying connection, it’s possible that it could get out of sync and not have ticked over when the network is back up, causing time skew but still having the machine allow network connections. Depending on the configuration, it could also plain shut down when it cannot get a connection and not restart when the connection is back.

                                            Saying that software crashes are unrealistic is in itself an unrealistic view in my opinion. Software WILL crash.

                                            Allowing clock skew to prevent connections to the machine definitely adds more moving parts to the remote access process and adds risk to that process failing.

                                    1. 2

                                      Just envision how complex would it be to deploy to mesos/kubernetes without the container abstraction…

                                      Nice and complete article!

                                      1. 4

                                        I was at Twitter when we deployed the first production deployment of mesos. It did not use “containers” as they are understood in the docker sense. We shipped binaries, there was a shared filesystem you could see if you logged into one of the machines your service was running on, and mesos set up service discovery to connect the local ports you were randomly assigned at boot time in an (iirc) env variable.

                                        For mesos, and, I believe, borg before it, containers came after to make things easier.

                                        1. 1

                                          containers came after to make things easier. That’s the whole point of it! Obviously you can do without, but like I said, imagine how complex it would be!

                                        2. 3

                                          Because those are built for containers. Containers came first. Maybe I am missing your point…

                                          1. 1

                                            I’m not speaking about the configuration details but the whole concept. Can someone point me to a technology that correctly and easily describe the runtime convention such as port binding, volume mounting, but as the same time enable extensibility and security?

                                            Sûre many tools exist to do things here and there, but none use them offer a coherent solution to build on top of.

                                            1. 2

                                              Sure, containers enabled this paradigm. But it’s not as if networking, data storage/access, scaling, and security didn’t exist before containers. They were just handled differently with different technologies.

                                              Does kubernetes make this easier/better? Probably depends who you ask. And for my take, you still have to run kubernetes on something. That something still has the same needs/requirements as always. (This is where you introduce the cloud layer, or someone else’s computer.)

                                              1. 3

                                                Does kubernetes make this easier/better?

                                                Similarly to how containers apply the benefits you can easily get from the Maven model to any runtime, I’ve always thought that kubernetes was just Greenspun’s Tenth Law but about the benefits of Erlang.

                                          2. 2

                                            Honestly, I think it wouldn’t be too bad.

                                            Artifacts would be shipped around as e.g. .zip files downloaded from HTTP servers, instead of containers pulled from container registries. Those artifacts would be tagged with metadata to indicate which runtimes they require, and nodes in the cluster would be tagged to indicate which runtimes they provide; the scheduler would schedule jobs based on those constraints. Kubernetes, at least, already supports this. On balance, I think dropping containers for this aspect of things would actually make the overall system simpler.

                                            Resource limits and namespaces would need to leverage underlying OS primitives directly, rather than going through the container abstraction. Probably a proto-container spec (like OCI?) would arise from these requirements naturally. This is where things would get more complex.

                                            1. 1

                                              In terms of a framework that would allow you to programmatically deploy (virtual) machines, get applications installed then configure those and start them, it’s basically what I built at my last job. I didn’t get onto any advanced features like automatic load distribution and automatic machine setups when the environment needs more resources but it was completely possible on the back of the basic features as well as the fact that the system did set up distributed (peer to peer and mastered) services together with their own encrypted virtual networks.

                                              I don’t know what other advanced features are possible in Kubernetes/Mesos that wouldn’t work without containers but AFAIK the security isolation is still better for VMs than containers and the networking should be easier since it’s simplified by not having a machine effectively be a router for the containers.

                                            1. 27

                                              And yes, a large part of this may be that I no longer feel like I can trust “init” to do the sane thing. You all presumably know why.

                                              No, I don’t. Please explain what you mean.

                                              Such appeal to everyone “knowing what you mean” and the implication that everyone supports your standpoint are toxic. They are a good way of making a personal opinion look like a group opinion. Combined with a fuzzy notion of “sane”, this is basically just spreading bile.

                                              And it works. If this mail were more complex or that sentence would be missing, it probably wouldn’t be here on lobsters. It’s certainly not posted for the review above it.

                                              1. 85

                                                If you’re reading the kernel mailing list, it can be assumed that you have some familiarity with the subject matter, and if not, you’re not missing anything crucial to the discussion here. Torvalds has decided not to point and name the implied party, probably to avoid another heated flame war on the mailing list.

                                                Some context to get you up to speed:

                                                He is referring to what is currently the most popular init system on Linux, systemd. systemd is a relatively recent development of Red Hat, and has been adopted by all major distributions. Prior to systemd gaining popularity, the init system was a hodgepodge of shell scripts, which clearly had its share of problems.

                                                However, systemd has been adding more features to its resume. Besides just being an init system, it has also absorbed the hardware abstraction layer udev, it implements its own dbus daemon (a popular Linux message bus used to communicate between different services and programs), it has taken control of some power management features such as suspend on lid close for laptops, it implements login and virtual terminal handling, it contains a dhcp client and server, and it provides its own system logger using a binary format that can in practice only be used through the tools provided by systemd.

                                                This attitude of trying to do everything from a single piece of software has proven to be somewhat controversial among Linux users, because the old UNIX mantra for software was “do one thing and do it well”. This is especially controversial for something as central as the init system, because it is always running and runs with elevated rights.

                                                The lead developer of systemd has also responded to a few issues with some unpopular comments, and has in the past been in conflict with the Linux kernel developers by refusing to cooperate on certain issues caused by Linux and systemd interaction. systemd has also, despite its widespread use, been hit by a number of fairly serious bugs, some of which had significant security impacts. The simplicity and potential impact of some of these bugs has left many people in doubt over the general quality of systemd and related projects.

                                                1. 18

                                                  In particular Linus has had specific issues in the past - there was a problem a while back where kernel developers would boot with the “debug” flag and systemd would start spamming the console with messages and drown out the kernel information said developers needed. See https://lkml.org/lkml/2014/4/2/580 where someone proposed a patch that would remove “debug” from /process/cmdline so that the presence of that flag was completely unavailable to userspace (including systemd), thus literally preventing the problem from happening. Real icky situation.

                                                  1. 9

                                                    Ha, while this isn’t quite what I called for, it is a great explanation of the state of things. “Controversial”, I certainly agree with (although, coming to systemd, I’m on the “It fixes a lot of things for me” side of things).

                                                    Thanks for that.

                                                    1. 14

                                                      There seems to be two main camps of users (3 if you include distro maintainers as a separate camp):

                                                      1. People who maintain a few systems, perhaps use Linux on the desktop.

                                                      2. People who maintain many systems.

                                                      Camp 1 people don’t mind when Systemd does something arbitrary, unexpected or indeterminate. Camp 2 people hate Systemd’s indeterminism.

                                                      Personally I hate magic. Systemd is magic. I can’t trust it to do what I want to do, only what it wants to do.

                                                      1. 9

                                                        As a person who maintains many systems professionally, i have to interject here since I always see it stated as fact that professional operators dislike systemd. I like systemd a lot because it gives badly needed structure to Linux service management. Most colleagues who worked with systemd feel the same. (This doesn’t mean it’s perfect or bug-free)

                                                        1. 4

                                                          Is there a reason you didn’t deploy daemontools or runit or some such to give badly needed structure to Linux service management before systemd forced it on you (however willingly)?

                                                          1. 5

                                                            I did use those at various times but that’s not the same as being the default that manages all services on the system. Systemd also has a powerful declarative configuration the other options did not.

                                                        2. 2

                                                          There seems to be two main camps of users (3 if you include distro maintainers as a separate camp):

                                                          1. People who maintain a few systems, perhaps use Linux on the desktop. Camp 1 people don’t mind when Systemd does something arbitrary, unexpected or indeterminate.

                                                          I’m a group 1 member but I absolutely hate when systemd doesn’t operate as I would expect a init daemon.

                                                      2. 4

                                                        This attitude of trying to do everything from a single piece of software has proven to be somewhat controversial among Linux users, because the old UNIX mantra for software was “do one thing and do it well”.

                                                        I’ve found this mantra to be only applicable in certain situations, usually when it comes to applications that users directly interact with. Things like email clients, text editors, and IRC clients (web browsers could spawn an entire discussion on this all their own). I’m not an expert on init systems, and your previous paragraph on systemd clearly shows its feature creep. But when it comes to an init system, I’ve always seen that as a complex process where it’s necessary for it to do more than one thing. This can be especially true with modern systems where everything you’ve mentioned (HAL, dbus, power management, login, networking, etc) being (arguably) necessary for the system to run correctly and in a useful way.

                                                        So, I wonder, is it possible to have an init system that is:

                                                        • fast
                                                        • effective
                                                        • correct

                                                        that still abides by “doing one thing, and doing it well”?

                                                        1. 8

                                                          The mantra predates text editors (well aside of ‘ed’), email and irc clients. Any user interfaces in the modern sense really. It meant using a bunch of small, single purpose programs (like cat, troff, tail, ps…) which could be combined by user to the desired effect with standard system mechanisms like redirection, pipes and shell scripts.

                                                          We can argue the practical merits of systemd forever but it’s fairly clear it goes against the tradition of UNIX systems development. It’s a huge, opaque, uncooperative beast that makes turtles cry. I hope Linus is close to the point where he’ll just come up with something more digestible.

                                                          1. 7

                                                            The original idea behind that mantra was to make tools that were:

                                                            1. simple (Note: I’m using the Rich Hickey definition here)
                                                            2. composable. This composability requirement is why all the command line tools ingest and output strings.

                                                            As systemd takes over more of a Linux machine, they destroy their own simplicity, requiring someone to keep a massive amount of state in their head to modify the code or even work on units as an administrator. However, they also destroy the composiblity of the system’s tools. Things like binary logs and internal-to-systemd protocols can’t be parsed by standard command line tools, and thus users lose this ability to compose different parts of the system. This has been my biggest issue with systemd, that it violates not only “do one thing and do it well”, but also the composiblity that makes that possible.

                                                            A side note on GUI’s: The GUI design model is specifically the opposite of “do one thing and do it well”. GUI’s are not designed for composibility, they are designed to take the user from one end of a specific process to the other. They trade off the ability to compose with other programs, for a more robust control of the user experience.

                                                            1. 1

                                                              What’s wrong with piping journalctl to things? Legitimately curious.

                                                          2. 1

                                                            The context of this discussion is around trying to bring sanity to rlimits for setuid processes…

                                                            In an attempt to provide sensible rlimit defaults for setuid execs, this inherits the namespace’s init rlimits:

                                                            $ ulimit -s 8192 $ ulimit -s unlimited $ /bin/sh -c ‘ulimit -s’ unlimited $ sudo /bin/sh -c ‘ulimit -s’ 8192

                                                            This is modified from Brad Spengler/PaX Team’s hard-coded setuid exec stack rlimit (8MB) in the last public patch of grsecurity/PaX based on my understanding of the code. Changes or omissions from the original code are mine and don’t reflect the original grsecurity/PaX code.

                                                            Certainly traditionally it has been trivially easy for a rogue daemon to bring a system to it’s knees…. since traditionally, out of the box, there are no rlimits imposed.

                                                            It is the init systems job to start daemons… it would be really nice if it imposes sane rlimits on anything it starts.

                                                            Systemd does that, and attempts to do it in sanish ways by imposing the limits on process groups. (ie. A rogue daemon cannot escape it’s constraints by spawning a legion).

                                                            I’m would be easily convinced that systemd’s approach is not the best and/or not correct.

                                                            However I’m certain that the linux ecosystem needs work in this area and systemd is at least undertaking that work.

                                                            1. 2

                                                              You seem to be mixing up system daemons and setuid utilities. This patch has nothing do with the limits systemd imposes on the processes it starts, so whatever systemd does, or other systems did not do, in this area is irrelevant.

                                                              There’s no question that systemd is capable of setting rlimits for child processes. The question is whether the limits systemd sets for itself are a good default template for setuid processes run by users.

                                                            2. 1

                                                              I figured it was systemd, and I know there’s a TON of systemd hate floating around, but I didn’t realize just how rampant the freeping creaturism had become.

                                                              That’s really unfortunate, thanks for the clarifying comment.

                                                              1. 4
                                                            3. [Comment removed by author]

                                                              1. 8

                                                                What qualifies as “insane”?

                                                                This might sound trollish, but I seriously don’t find “sane/insane” distinctions very useful (beyond their connotations). In my experience, these words get used when you can’t construct a better argument for why the other side is doing something wrong, so you call yours “sane” and the others “insane”.

                                                                1. [Comment removed by author]

                                                                  1. 8

                                                                    That sounds like a nice definition, but “without reason or logic” is just as fuzzy, “counter productive, destructive, harmful” are also easily stated, but must be followed by hard facts to be held up.

                                                                    Also, regarding the principle of least surprise: Matz (who popularised it) also famously said that it applies to his surprise.

                                                                    Your definition just moves the playing field.

                                                                    Also, I would argue that anyone implementing a piece of software so central to the Linux world is a “domain expert”. This is boundary play at its finest.

                                                                    1. 19

                                                                      For a specific example: How would you classify the change to kill tmux servers after a user logged out? Lots of people found that surprising. And in the larger space of existing init systems, quite unprecedented. I think “insane” is lacking precision, but adequately captures many people’s sentiment.

                                                                      1. 10

                                                                        and to add to this response, I find it insane that the response was asking tmux to include a change for the new behavior systemd enforced.

                                                                        1. 2

                                                                          Especially since it changes how every unix has behaved for almost 30-40 years regarding HUP.

                                                                          Or a more recent one where it parses a username of “0haha” as being an invalid username and runs the unit as root. And now perfectly valid usernames starting with a number won’t work in systemd unit files as they get interpreted as being invalid because systemd can’t seemingly parse numbers in a config file sanely to distinguish a user name versus a user id.

                                                                          This all might sound like splitting hairs, but breaking userspace (HUP behavior shouldn’t need a patch for your init in tools), and not parsing a username sanely are pretty basic things I would expect a first year undergrad to be able to do.

                                                                          So yes, I agree insane is a good word to use for things. I could come up with hard facts, but systemd really feels like one step forward and two steps back for a lot of things. I don’t really feel like its a very good example of good engineering practices, aka binary logs that can be corrupted forcing you to do insane things to get a system online due to a short write to the filesystem is… also insane, we have decades of knowledge of how to do this that has been ignored.

                                                                          If the corners aren’t rounded on this desk, why should I feel safe about the rest of the desk?

                                                                      2. 2

                                                                        Boundary play? The guy who wrote Linux made an opinion about it being so.

                                                                    2. 13

                                                                      In my experience, these words get used when you can’t construct a better argument for why the other side is doing something wrong, so you call yours “sane” and the others “insane”

                                                                      “In your experience”, huh? So you’re just extrapolating your own personal experiences to all of mankind, then? “Citation needed!”

                                                                      You see, it’s easy to filibuster any conversation by calling for better argumentation, proof, evidence, studies to back claims up and so on.

                                                                      You know perfectly well why someone might call systemd “insane”. What’s your actual contribution to the conversation, besides signalling to everyone what a rational and sophisticated person you are?

                                                                      1. 10

                                                                        “In your experience”, huh? So you’re just extrapolating your own personal experiences to all of mankind, then? “Citation needed!”

                                                                        No, I don’t. That’s why I wrote: In my experience.

                                                                        You know perfectly well why someone might call systemd “insane”.

                                                                        No, I don’t. I use systemd every day and I’m very fine with how it works and how it behaves.

                                                                        It has, as all implementations of a thing, issues and flaws, but that’s all. I’d be happy to try an alternative, which would for sure improve in a lot of areas (and may be worse in others), but that’s a trade-off, nothing more.

                                                                        What’s your actual contribution to the conversation, besides signalling to everyone what a rational and sophisticated person you are?

                                                                        I’m highlighting a conversational pattern that is all too often used to create unity were there is none. I’m neither rational or sophisticated.

                                                                1. 10

                                                                  To be honest, I’m curious about the demographic breakdown of the applicants for speaking at the conference.With 3%, the demographic could have just not at all applied in the first place, making it extremely difficult to select it.

                                                                  1. 15

                                                                    I thought I remember hearing that it was a double-blind selection process – which is generally considered a Good Thing (tm). The problem is that when you have only a small number of boxes to fill, it’s easy for the margin to be swept out in a double-blind system. I don’t necessarily think that there’s wrong-doing in the case that they did follow a double-blind selection method. They declared a process, followed it, and got a result they didn’t like; but was ‘true’ to the process in some sense.

                                                                    I can see why there was disappointment in that result, but I don’t think the takeaway should be “This process is fundamentally flawed” – the process is very fair (it chose in a way that modeled the distribution, if you have 100’s of submissions, and 10’s of slots, 3% is likely <1 total talk), but in this case your goal isn’t to be fair, it’s to encourage the distribution to go up. You need, in some sense, to take an affirmative action[1] to ensure that the minority gets some minimum of representation.

                                                                    What might be better in conference planning, then, is to have a portion (say, 75%) of the talks be chosen blind, and the remainder split absolutely equally based on diversity characteristics (race, gender identity, etc). Do not publicly disclose which talks are which, this allows a conference organizer to get a good sense of actual representation (by examining the 75% after they’ve been picked), and also ensure that some minimum amount of diversity in speakership is achieved. The benefit of this approach is that that minimum amount will always ensure that leaders from the minority community[2] is represented, which encourages other members of that community to engage and generally increase the number of people belonging to that group. As the sample rates change, you can start to assign fewer ‘reserved’ seats to that minority, since they will be accounted for accurately in the blind process (assuming all else is equal).

                                                                    I think this is a growing pain, ElectronConf was using a blind selection process, but found it insufficient. That’s a good thing. They postponed the event, that’s an unfortunate thing, but indicates they value diversity more highly than the disruption they caused, that’s also probably a good thing, or at least neutral. I think that as they, and conferences like them, go forward, they’ll find ways to balance the host of concerns around ensuring that underrepresented subgroups get to participate in a way that positively effects the community. I think the thing that will be hardest to realize in these processes is the fact that being diverse and being fair are, in some sense, opposed to one another in this situation. Being 100% biased toward fairness will result in a net zero change to the makeup of the community, it is a male-dominated industry, and when you sample fairly, you’ll get a male-dominated result. If you really want something to be diverse, you have to ‘tip the scale’ towards it, and that’s a really hard thing for people (especially, it seems, people in this industry) to accept.

                                                                    [1] I believe I’ve just said some words that might make people mad, I trust lobste.rs will react better than most places, though.

                                                                    [2] Because if you’re speaking at a conference, you are a de facto leader of a community, perhaps not the leader, but a leader for sure.

                                                                    1. 3

                                                                      I agree that you have to selectively bias in favour of the minority/minorities to actually put them in the spotlight, however I appreciate the detailed explanation. My point was that there could have simply been 0% of women applying which would’ve meant 0% of women being selected even if they actively tried to select women.

                                                                      I don’t agree that you need to be a leader of any kind to speak. While my previous employment was a 100% white male they did in my opinion get something right in that they encouraged everyone to present. It didn’t matter whether you were the best in the subject matter on the team or not, everyone could teach someone else something new.

                                                                      On the back of the last point, it would seem to me (personal opinion) that encouraging non-leaders to speak would help even further with the diversity at conferences. Minorities entering the field are (again, in my opinion) even less likely to be leaders than to have chosen the profession in the first place, so encouraging “fresh” people to speak would in the least get their point of view heard, if not teach people new things or ways of thinking.

                                                                      1. 11

                                                                        If 0 women applied then they may need to put ore effort into encouraging women to apply if they care about having a diverse lineup.

                                                                        1. 4

                                                                          If they aren’t actively discriminated against, which I doubt github does, then it is up to these individuals to apply and take the final step. You can lead a horse to water, but can’t force them to drink.

                                                                          1. 4

                                                                            If Github wants a more diverse lineup, they’ll need to be more proactive about it, since the status quo is clearly not going to get them there.

                                                                            1. 4

                                                                              doesn’t work that way; women tend to estimate their abilities lower, largely due to the societal mental picture of a good programmer as male, white/asian and young. so if you just put out a generic call for applications, you’re going to get them from people who have spent their lives being told they have stuff worth speaking about.

                                                                          2. 3

                                                                            My wording was inartful wrt ‘leaders’ – my point was that by speaking, on some level, you become a leader. The way to leadership is by leading, and speaking at a conference is in some way leading people (in the case of a conference talk, it may simply be leading towards a technical end or a social one, but it is leadership of some form).

                                                                            I don’t think that you need to go looking for leaders, I think you make them by virtue of giving them a platform.

                                                                            Definitely there is a possibility that 0% of women (in this case) applied, I think that you’d have to seek out women directly for that subset of talks positively biased (i.e., pull in speakers, rather than having speakers pushed toward you.

                                                                            Generally though, I think we’re in agreement, your last paragraph better says what I intended wrt the ‘leadership’ stuff, bringing in people who are not beforehand leaders is definitely a good thing.

                                                                      1. 8

                                                                        … or just change the password from the default one

                                                                        1. 1

                                                                          Forcing users to change the password is needed. If you allow the users to shoot themselves in the foot, they’ll do it and blame you.

                                                                          1. 1

                                                                            yeah, but the “Update YOUR Pi” headline is very misleading, all you need to do is change the pw and you’re secured

                                                                        1. 5

                                                                          I usually use vertical tab in shell pipelines where other common delimiters (tab, colon, comma) are likely to be in the input. It allows some robustness without excess crazyness.

                                                                          1. 13

                                                                            That’s what GS, RS, and US are for! Group, Record, and Unit Separator, ASCII 1D, 1E, 1F.

                                                                            1. 7

                                                                              So, I always get the feeling that the lower values in ASCII are Black Magic ™, not meant for mere mortals.

                                                                              But, as I’ve gotten older, I’ve seen more and more things that actually worked really well and then were abandoned because people forgot what problems they solved–only to reinvent those solutions later.

                                                                              So, where can we learn about the utility and history of all those goofy control codes?

                                                                              1. 3

                                                                                To be fair GS, RS and US are only useful if you’re encoding text and only ASCII or UTF-* or something ASCII based that is either fixed width or doesn’t use something under 1F as a prefix and your data doesn’t use 1D, 1E or 1F.

                                                                                However GS RS and US are still alive and kicking if you work with Mainframes because their database dumps (which are the cleanest ever) use that instead of CSV/SQL. But the best ‘history’ of control codes right now is learning about the VT100 and VT220 standards.

                                                                            2. 1

                                                                              That actually gave me an idea on what to do if my shell scripts' input delimiters start being problematic and need to be worked around to accept input that has them.

                                                                              The solution might not be to drop something that isn’t being used but to start using it more because it does have a purpose.