Eh. This focuses on the distinction between Linux and Unix, and argues that Linux winning doesn’t mean that Unix wins. But honestly, I’d think the bigger issue is that a majority of Linux systems no longer have a Unix-like userland (because they are running Android or an embedded environment). IMO, that difference vastly eclipses even the difference between BSD init and systemd.
(I used HP/UX, SunOS, Solaris, A/UX, and 386BSD before Linux existed, and I worked as a Linux sysadmin in the pre-systemd era. Systemd is good, actually. It solves problems that need solved, and would otherwise be solved in crufty and inconsistent ways. Systemd being good doesn’t mean something better won’t come along, either.)
It’s also ahistorical, comparing Linux which has seen decades of development against a static target like POSIX.
The reality is that POSIX is 33 years old and has been virtually unchanged for almost 20 years now. Of course Linux looks like it’s shrugging off POSIX. That’s just a consequence of the fact that we’ve learned a lot more about computing in 33 years and computing itself has also radically changed.
Much of what Linux is doing is fixing serious shortcomings in POSIX. The async IO interface in POSIX isn’t useful. They never bothered with anything graphics-related. It’s clear that some system-wide messaging capability or a more flexible IPC method is needed, but that doesn’t exist in POSIX (every modern OS has adopted some solution to this like dbus). POSIX is woefully out of date when it comes with dealing with non-English speakers. Pretty much 100% of people that work on filesystems complain about the semantics of fsync. The “everything is a file” concept turned out to not be a great abstraction after all for many tasks. The POSIX IO consistency model is too strong, we’re hitting the limits of what is possible with it.
We could go on and on. The two will continue to diverge and that’s not at all surprising. Eventually they will diverge so far that POSIX will need an update that surveys the common components between *nix like systems again.
Systemd enables dynamic hardware and dynamic routing of data in software based on hardware events. As a whole concept, this is what’s needed for anything not a single application server (example: your desktop or laptop). There’s glaring flaws in systemd, but we can do better. Removing and going back to init.d runlevel scripts is not the way to do that!
I don’t intend to argue whether systemd is good to bad but there’s a huge gap between init.d scripts and systemd. The combination of udev and dbus enabled dynamic routing of data in software based on hardware events for years before systemd was invented. That happened on systems that relied on init.d scripts. They are also core parts of systems that rely on systemd.
That detail is helpful, and I wish OP had more such nuggets. After reading OP I found myself thinking, “yes, Linux is different. But why is it worse?” Why should POSIX matter to more people? Are udev and dbus reinventing Unix poorly, or are they aiming for something new that the designers of UNIX hadn’t designed for?
It’s common for people to have divergent priorities. In this case, are they misguided?
The author literally doesn’t mention systemd at all. So you’re now projecting that into the article.
I do like that tragedy of systemd talk by the BSD guy (I think I saw the BSD Canada conference version). It’s good and a system layer can be good. Even prior to systemd, Linux was slowly developing a system layer. Redhat HAL went away and we were left with dbus + networkmanager + cups and host of little tools that could be strung together.
Yes systemd is modular, but you can’t really replace any of the little modules. The target file structure is nice, but it’s expanded way out from just process management to encompass other parts of the systems, instead of just hooking into any of the previous system layer tools.
But I digress, I don’t think the author is really talking about systemd at all. If it’s included, it’s auxiliary. A lot of people quote the DOTADOW philosophy statement without the 2nd part:
Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
Programs should work together. I think dbus goes in the right direction, and you can run it anywhere. But now look at some more modern Linux specific things: Docker.
Docker only runs on Linux. Docker for Mac and Windows run in hypervisors. It’s dependent on cgroups, namespaces and lots of specific Linux things. Can you implement a container system in BSD that’s compatible with the Docker API, but uses ZFS for layers and jails for chroots instead? Absolutely, and there are some attempts at it being made. But I think it goes to the point of the article: certain things are now tied deeply into Linux and cannot be run on other UNIX like (macos/darwin, freebsd, etc.) systems.
The author literally doesn’t mention systemd at all. So you’re now projecting that into the article.
from the article:
This leaves Linux, a system Dennis Ritchie described in 1999 as “draw[ing] so strongly on the basis that UNIX provided”. My worry is Linux and the community around it have strayed further from UNIX for a long time since, to the point where systemd discussions include comments like this with hundreds of upvotes:
systemd or not/BSD vs. Linux is an incredibly minor thing compared to the actually significantly different systems that have been mulched in Unix’s path (i.e. VMS).
What GNU wanted to be was much more akin to ITS than to Unix, I believe. GNU Emacs and GNU info have direct ancestors on ITS. GNU su’s not requiring membership in a specific wheel group is analogous to the user switching mechanisms in ITS (where strictly enforced administrative roles were apparently déclassé). The GNU tools have never been about “doing one thing well”; they’ve always been kitchen sinks compared to classic Unix.
None of this is objectively bad, of course, just pointing out that “GNU” does in fact mean “GNU’s Not Unix.”
I don’t think “winning” is the right choice of words here. Rather, I believe the world has moved on from UNIX.
Considering that UNIX was an operating system developed in the 1970s, this seems pretty reasonable to me. There have been many developments in both hardware/software technology as well as development practices that have obviated many of the operating systems and programs of yesteryear. UNIX wasn’t designed to be run on embedded devices, or laptops, or TV set-top boxes, because none of those things existed yet. We’ve been living beyond the constraints that UNIX was built within for a while now, and there have been some great developments in recent years that improve the experience on a UNIX-based OS.
Personally, after years of using Upstart on Ubuntu, dealing with all the issues of legacy init scripts interacting with Upstart jobs, and things being quite hard to manage on multiple OSes, I’m a huge fan of how systemd has made a lot of that work much simpler. Its documentation is better than the myriad of tools that only exist in manpages and developer’s brains, it’s easier to get the information you need out of systemd’s machinery, and it’s a lot easier to work with than launchd because you don’t have to keep esoteric XML syntax in mind.
All in all, I’m a big fan of systemd and the changes it’s introduced to the world of Linux. I wish BSD and other UNIX-based OSes would adopt it or something like it, even if they feel like it should be rewritten to support native OS features, because it’s truly a great idea and helps Linux stay competitive in a world that free software could have never predicted.
Basically the point is that POSIX is out of date, and Android, OS X, etc. all rolled their own mechanisms for essential services like IPC, and they don’t use POSIX APIs.
There was an additional HN discussion at Posix has become outdated. Ideally the unused or rarely used POSIX APIs would be removed to reduce the attack surface.
I’ve got a draft that’s working towards a similar point (that is, POSIX defines existing stuff that’s no longer relevant for modern application design, more for people like me to rules lawyer people about AIX)
Speaking from a world far away from Unix: Your work is going to be making Linux or at least POSIX-specific programs working on your system or setting up APIs so work can be moved to Docker containers as appropriate.
Depends on what you understand by “work”. I personally am barely getting by, but at least I’m doing work I’m happy with rather than forcefully working on something that’s doesn’t fit me.
About being happy that Windows didn’t win. I hope this isn’t taken as a purely grumpy rant, however I think that we see concepts paralleling the complexities of the Windows world start to take hold in the ways that people and companies use Linux today. Large parts of where cloud environments and technologies especially from/sponsored by RedHat and CNF are common resemble mindsets that at least in the 2000s were more predominant in the Microsoft/Windows ecosystem and not in BSDs, Solaris, and most Linux distributions (maybe in some degrees in RHES and SLES, but I think even there it started later).
Some of the developments seemn like a rerun. Also looking at cloud providers. First the virtual server was reinvented as compute instances and then managed web hosting gets reinvented as server less. Cloud Functions come close to CGI and PHP.
The Linux on the desktop movement pushed a lot of copying from Windows and now even servers frequently use dbus (the D stands for desktop). At the same time Windows went the opposite direction. Starting with Interix around 2000 Windows became POSIX compatible. At least to some degree.
I also think targeting UNIX/POSIX was kind of replaced with targeting Windows/Linux (Ubuntu mostly)/MacOS, then maybe accepting patches sometimes for others. Nobody really expects something else. Docker goes a step further, usually even kicking the largest part of Linux out. But maybe that will get replaced with WASI at some point, might be the same solution when it’s all the same anyways.
In other words I think it’s easy to fool yourself thinking Unix or even Linux won. If you only speak of system calls that might be true to some degree, but I don’t think that’s how it’s meant.
Eh. This focuses on the distinction between Linux and Unix, and argues that Linux winning doesn’t mean that Unix wins. But honestly, I’d think the bigger issue is that a majority of Linux systems no longer have a Unix-like userland (because they are running Android or an embedded environment). IMO, that difference vastly eclipses even the difference between BSD init and systemd.
(I used HP/UX, SunOS, Solaris, A/UX, and 386BSD before Linux existed, and I worked as a Linux sysadmin in the pre-systemd era. Systemd is good, actually. It solves problems that need solved, and would otherwise be solved in crufty and inconsistent ways. Systemd being good doesn’t mean something better won’t come along, either.)
It’s also ahistorical, comparing Linux which has seen decades of development against a static target like POSIX.
The reality is that POSIX is 33 years old and has been virtually unchanged for almost 20 years now. Of course Linux looks like it’s shrugging off POSIX. That’s just a consequence of the fact that we’ve learned a lot more about computing in 33 years and computing itself has also radically changed.
Much of what Linux is doing is fixing serious shortcomings in POSIX. The async IO interface in POSIX isn’t useful. They never bothered with anything graphics-related. It’s clear that some system-wide messaging capability or a more flexible IPC method is needed, but that doesn’t exist in POSIX (every modern OS has adopted some solution to this like dbus). POSIX is woefully out of date when it comes with dealing with non-English speakers. Pretty much 100% of people that work on filesystems complain about the semantics of fsync. The “everything is a file” concept turned out to not be a great abstraction after all for many tasks. The POSIX IO consistency model is too strong, we’re hitting the limits of what is possible with it.
We could go on and on. The two will continue to diverge and that’s not at all surprising. Eventually they will diverge so far that POSIX will need an update that surveys the common components between *nix like systems again.
This appears to be an opining for the “Old Times”, and a hit piece against systemd.
The opining for the old times of Unix reminds me of the quotes by Socrates that can be summed up as “Kids these days”.
As for Systemd, thats a harder discussion topic, but I think https://www.youtube.com/watch?v=o_AIw9bGogo is a good way to approach that topic.
Systemd enables dynamic hardware and dynamic routing of data in software based on hardware events. As a whole concept, this is what’s needed for anything not a single application server (example: your desktop or laptop). There’s glaring flaws in systemd, but we can do better. Removing and going back to init.d runlevel scripts is not the way to do that!
I don’t intend to argue whether systemd is good to bad but there’s a huge gap between init.d scripts and systemd. The combination of udev and dbus enabled dynamic routing of data in software based on hardware events for years before systemd was invented. That happened on systems that relied on init.d scripts. They are also core parts of systems that rely on systemd.
That detail is helpful, and I wish OP had more such nuggets. After reading OP I found myself thinking, “yes, Linux is different. But why is it worse?” Why should POSIX matter to more people? Are udev and dbus reinventing Unix poorly, or are they aiming for something new that the designers of UNIX hadn’t designed for?
It’s common for people to have divergent priorities. In this case, are they misguided?
The author literally doesn’t mention systemd at all. So you’re now projecting that into the article.
I do like that tragedy of systemd talk by the BSD guy (I think I saw the BSD Canada conference version). It’s good and a system layer can be good. Even prior to systemd, Linux was slowly developing a system layer. Redhat HAL went away and we were left with dbus + networkmanager + cups and host of little tools that could be strung together.
Yes systemd is modular, but you can’t really replace any of the little modules. The target file structure is nice, but it’s expanded way out from just process management to encompass other parts of the systems, instead of just hooking into any of the previous system layer tools.
But I digress, I don’t think the author is really talking about systemd at all. If it’s included, it’s auxiliary. A lot of people quote the DOTADOW philosophy statement without the 2nd part:
Programs should work together. I think dbus goes in the right direction, and you can run it anywhere. But now look at some more modern Linux specific things: Docker.
Docker only runs on Linux. Docker for Mac and Windows run in hypervisors. It’s dependent on cgroups, namespaces and lots of specific Linux things. Can you implement a container system in BSD that’s compatible with the Docker API, but uses ZFS for layers and jails for chroots instead? Absolutely, and there are some attempts at it being made. But I think it goes to the point of the article: certain things are now tied deeply into Linux and cannot be run on other UNIX like (macos/darwin, freebsd, etc.) systems.
The author mentions systemd a couple of times. It seems to be an example he’s using to support his argument.
oh huh, I did miss it, or maybe it didn’t register. It’s mentioned twice it seems.
from the article:
(Bold is my addition)
I know. I already addressed that in the above comment:
https://lobste.rs/s/ezqjv5/i_m_not_sure_unix_won#c_leook6
systemd or not/BSD vs. Linux is an incredibly minor thing compared to the actually significantly different systems that have been mulched in Unix’s path (i.e. VMS).
Interesting historical note, Windows NT can be considered the next generation of VMS.
What GNU wanted to be was much more akin to ITS than to Unix, I believe. GNU Emacs and GNU info have direct ancestors on ITS. GNU
su’s not requiring membership in a specific wheel group is analogous to the user switching mechanisms in ITS (where strictly enforced administrative roles were apparently déclassé). The GNU tools have never been about “doing one thing well”; they’ve always been kitchen sinks compared to classic Unix.None of this is objectively bad, of course, just pointing out that “GNU” does in fact mean “GNU’s Not Unix.”
I don’t think “winning” is the right choice of words here. Rather, I believe the world has moved on from UNIX.
Considering that UNIX was an operating system developed in the 1970s, this seems pretty reasonable to me. There have been many developments in both hardware/software technology as well as development practices that have obviated many of the operating systems and programs of yesteryear. UNIX wasn’t designed to be run on embedded devices, or laptops, or TV set-top boxes, because none of those things existed yet. We’ve been living beyond the constraints that UNIX was built within for a while now, and there have been some great developments in recent years that improve the experience on a UNIX-based OS.
Personally, after years of using Upstart on Ubuntu, dealing with all the issues of legacy init scripts interacting with Upstart jobs, and things being quite hard to manage on multiple OSes, I’m a huge fan of how
systemdhas made a lot of that work much simpler. Its documentation is better than the myriad of tools that only exist in manpages and developer’s brains, it’s easier to get the information you need out of systemd’s machinery, and it’s a lot easier to work with than launchd because you don’t have to keep esoteric XML syntax in mind.All in all, I’m a big fan of
systemdand the changes it’s introduced to the world of Linux. I wish BSD and other UNIX-based OSes would adopt it or something like it, even if they feel like it should be rewritten to support native OS features, because it’s truly a great idea and helps Linux stay competitive in a world that free software could have never predicted.By not defining “Unix”, this is question-begging.
Hm I’m trying to find an old comment I made on this paper, but Google and lobste.rs search are failing me:
POSIX Abstractions in Modern Operating Systems: The Old, the New, the Missing http://nsl.cs.columbia.edu/papers/2016/posix.eurosys16.pdf
Basically the point is that POSIX is out of date, and Android, OS X, etc. all rolled their own mechanisms for essential services like IPC, and they don’t use POSIX APIs.
2016 comments:
https://lobste.rs/s/jhyzvh/posix_has_become_outdated_2016#c_vthri6
https://news.ycombinator.com/item?id=11652609
There was an additional HN discussion at Posix has become outdated. Ideally the unused or rarely used POSIX APIs would be removed to reduce the attack surface.
I’ve got a draft that’s working towards a similar point (that is, POSIX defines existing stuff that’s no longer relevant for modern application design, more for people like me to rules lawyer people about AIX)
Well, and I’m not sure that it matters wether it did or not.
I personally, am not interested in winning/losing, who wins or who loses.
I’m interested in what technology stacks I’m going to be forced to deal with, due to their ubiquity, over the coming decade.
What and how would force you?
Work. Try to find a systems engineering job that doesn’t work with Linux.
They exist, but are rare and getting rarer.
Speaking from a world far away from Unix: Your work is going to be making Linux or at least POSIX-specific programs working on your system or setting up APIs so work can be moved to Docker containers as appropriate.
Depends on what you understand by “work”. I personally am barely getting by, but at least I’m doing work I’m happy with rather than forcefully working on something that’s doesn’t fit me.
About being happy that Windows didn’t win. I hope this isn’t taken as a purely grumpy rant, however I think that we see concepts paralleling the complexities of the Windows world start to take hold in the ways that people and companies use Linux today. Large parts of where cloud environments and technologies especially from/sponsored by RedHat and CNF are common resemble mindsets that at least in the 2000s were more predominant in the Microsoft/Windows ecosystem and not in BSDs, Solaris, and most Linux distributions (maybe in some degrees in RHES and SLES, but I think even there it started later).
Some of the developments seemn like a rerun. Also looking at cloud providers. First the virtual server was reinvented as compute instances and then managed web hosting gets reinvented as server less. Cloud Functions come close to CGI and PHP.
The Linux on the desktop movement pushed a lot of copying from Windows and now even servers frequently use dbus (the D stands for desktop). At the same time Windows went the opposite direction. Starting with Interix around 2000 Windows became POSIX compatible. At least to some degree.
I also think targeting UNIX/POSIX was kind of replaced with targeting Windows/Linux (Ubuntu mostly)/MacOS, then maybe accepting patches sometimes for others. Nobody really expects something else. Docker goes a step further, usually even kicking the largest part of Linux out. But maybe that will get replaced with WASI at some point, might be the same solution when it’s all the same anyways.
In other words I think it’s easy to fool yourself thinking Unix or even Linux won. If you only speak of system calls that might be true to some degree, but I don’t think that’s how it’s meant.