1. 34
  1.  

  2. 36

    Personal opinion: The GNU tools are superior to all other implementations for the USER. It is called “userland” because it is about the users, not about some mysterious POSIX compliance. I even install them on macs, because they are just better. I do not care one bit about the FSF or RMS or whatever, I use the GNU tools because they are better. I think back to the dark times I had a Solaris 8 box at work and everything was oh-so-POSIX compatible. Thank you, but no thank you.

    1. 23

      Yeah, what this author seems to be unaware of is that POSIX does not say that you can’t implement extensions, and in fact that’s basically expected. POSIX codifies things that are already widespread practice. Features go into code and then into POSIX, not the other way around. If you try and get some new feature into POSIX, they’ll tell you to go away because it isn’t in any implementations yet. So this:

      POSIX certified means that these OSs not only comply with POSIX but don’t add any compatible extensions. GNU, on the other hand, is POSIX compatible, but adds extensions (e.g. long options, options and behaviors that aren’t defined by the specs, etc.)

      is not correct, for multiple reasons. POSIX certification does not mean that you can’t add extensions. On the flip side, GNU is not POSIX compliant! The mere existence of the POSIXLY_CORRECT environment variable proves this.

      I found that the further I got into this article, the more inaccuracies like this there were. By the end I was pretty frustrated because it appeared that the author was speaking with authority (and a poorly hidden emotional undertone) on a subject that they actually were not an authority on.

      1. 13

        Indeed, the post links to Drew DeVault’s post about glibc isalnum segfaulting, and the pointed glibc weirdness is there exactly because of localisation, which is falls under the umbrella of, usability.

        The GNU tools put emphasis on usability above all else and that is a good thing. Alpine, borne out of BusyBox, doesn’t even come close. Granted, Alpine/BusyBox have their uses, places where humans aren’t often needed, but on user machines the GNU userland is superior.

        1. 4

          The linked article was discussed here.

          1. 3

            Alpine, borne out of BusyBox, doesn’t even come close

            Busybox doesn’t seem so bad these days; I’ve ran some systems on busybox, and I have to say I hardly notice the difference. It implements POSIX and most useful extensions. My biggest gripe is that it doesn’t allow configuring the ls colours without a recompile (the defaults don’t work well on light backgrounds).

            1. 1

              Update for future readers: I recently did a tail -n100000 with a fairly large 2G log file on a busybox system, and Busybox tail is horribly slow; this took over 1 hour(!!)

              GNU find is nearly instantaneous. I wrote a quick naïve implementation in Go and that took about a second (give or take) with no optimisations what-so-ever. I didn’t check why Busybox is so slow, but Busybox wants to be small and sometimes this trades off performance. Sometimes severely so.

            2. 1

              These days human usability is mostly about the GUI though. And the corresponding high-level tooling. For low-level tools, portability/simplicity/consistency are becoming bigger concerns. Who even expects all the localization things to be in libc?! I always thought that any localization stuff is the domain of libraries like ICU and standard libraries of higher-level languages, it never even occurred to me to look in libc for that. Every time a low-level library tries to do that kind of thing, it just results in nasty surprises like the infamous “Turkish İ” problem.

              1. 10

                from that comment I would guess that your native language is English, so you have no use for localisation in your shell. For many, many people that is not the case and they benefit greatly from the support of even the lowest level tools. It makes it more accessible to more people.

                1. 6

                  As was discussed on this site a year ago (see my comment above), glibc supports locales because locales are part of the C89 standard.

                  You might design your software to move locales support “up the stack” but glibc does not have that luxury.

                  And a libc that does not support locales is strictly speaking not standards-compliant, hence probably not portable.

                  1. 6

                    Depends on how you use your computer I guess. I do most things from the shell, so usability of the shell and shell tools pretty much is usability for me. This is of course not how most people use their computer these days, but there’s still a fairly decent number of us, and when GNU started out it’s how the majority of people used their computers.

                    The shell has always served two somewhat conflicting purposes: scripting and direct usage.

                  2. 1

                    It’s ironic that you use localisation as an example because I recently got bitten by GNU cut not actually being encoding-aware: the --characters and --bytes options do the exact same thing! BSD cut doesn’t have this problem.

                    1. 2

                      Pretty sure the -c option is spec’d by posix to be bytes.

                      1. 1

                        No, it’s specified to be characters. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/cut.html

                        There’s even a -n option for not splitting multibyte characters that GNU cut straight up ignores.

                  3. 11

                    Yeah the statements about POSIX in this post are silly – one of many serious misunderstandings. This is a low quality post.

                    POSIX certified means that these OSs not only comply with POSIX but don’t add any compatible extensions.

                    Huh? That’s not what POSIX means. Also, POSIX is generally old, and outdated, and people need to do things with their computers. It makes zero sense to implement only POSIX. This is a weirdly religious idea.

                    A recent example I gave is that GNU awk supports match() for submatch extraction, but original awk, mawk, and POSIX don’t. You should use this; it will make your awk scripts better.

                    Another one is the paper about how modern Unixes don’t actually use POSIX for IPC and such because it doesn’t do what they need (binder on Android, DBus on Linux, something else on OS X).

                    Open source extensions are good because they’re … open source. Vendor lock-in is bad, but the best you can do is give the user the right to modify, fork, redistribute, etc.


                    That said, I do think GNU code is “inexplicably bad” (a statement I heard somewhere else). There are many collaborative open source projects like CPython, LLVM, etc. which have higher code quality, not to mention the ones done by fewer authors (sqlite, Lua, etc.)

                    1. 7

                      My reasons for disliking GNU are the exact opposite of this article: I don’t think they do nearly as much as they could to improve UX. It’s “good enough” I guess, but that’s not the same as “good”.

                      POSIX was always intended as a “minimum feature set that works the same everywhere” type of specification, originating during the great Unix wars. It was never and will probably never be at the forefront of innovation. It’s a dictionary: it recorded what “language” people were using.

                      These days I just write most scripts in /bin/zsh, because it’s just so much more useful. Want to run it on OpenBSD? No problem, install zsh! Why zsh and not bash? Because zsh is clearly the better shell in almost every way and always has been really. A lot of “bash features” were added to zsh first (and both of course borrowed liberally from the then-proprietary ksh) and the only reason bash seems to be the de-facto default is “because GNU”.

                      The article complains about head adding -c. The way head and tail works is just weird, I can never remember how it works. If I were to write a “coreutils” package then I’d add POSIX head and tail for compatibility, and then move on to a much more useful lines or something which has a UX I can actually remember and which includes all the features from head and tail except better.

                      Or in short: POSIX is stuck in 1980. GNU is stuck in 1990.

                      1. 4

                        Personally, I used non-POSIX commands such as head --bytes N (the example in the article) from the shell so many times and they saved me a great deal of time.

                        Would I use head --bytes N in a script that is going to be published? No. I’ll use cut -b -N or something that is, if not 100% POSIX compliant, at least widely compatible.

                        Am I happy that non-POSIX features such as head --bytes N exist? Yes. I can use them while prototyping or in some quick-n-dirty pipeline.

                      2. 11

                        Why are we vendor locked into GNU? Why are we still pretending that’s good or not as evil as the vendor lock-in measures taken by Google, MS, or Apple?

                        Because FSF’s explicit values are better aligned with my personal interests and their software implementations have so far proved trustworthy and in line with those values?

                        1. 10

                          The problem with this is that if you unknowingly rely on GNU extensions you might believe your scripts will work anywhere, but might not. There’s nothing bad in using them if you know you’re using them. And that’s precisely the problem I see with many GNU fans: they more often than not are totally unaware of the portability issues GNU poses… and how “learning GNU” isn’t exactly “learning how to use UNIX.”

                          I don’t see how this is related to GNU at all. I see this in various places:

                          • C++ code doesn’t compile under all C++ compilers, because it is using Visual Studio’s C++ extensions,
                          • E-mail attachments don’t open under all e-mail clients, because they’re being sent in some Microsoft proprietary format,
                          • Word documents are being shared between people, but nobody asks if everyone uses MS Office,
                          • Some page doesn’t work in all browsers, because the page uses some Chrome-only API,
                          • other examples when searching for “Embrace, Extend, Extinguish”.

                          I think that the absence of knowledge is a natural thing, and it’s better to spread the knowledge how to build software properly, instead of trying to limit the technology we use to some least common denominator in fear that someone one day will want to use our software in some restricted environment, and THEN it won’t work.

                          1. 5

                            I believe the author’s point is that the examples you gave above are different from GNU because everyone hates them, and most people give GNU a pass when their code results in similar situations.

                            I don’t think it’s a very good argument, because as the author even points out, you have the freedom to run GNU code on your machine, so if you find a piece of software that’s “not portable” according to the POSIX standard, it doesn’t prevent you from running it the same way that getting a document in MS Word format does.

                          2. 9

                            This is a very strange post from someone who seems to have quite a hatred of GNU, so it is more of a crusade.

                            The fact is that many of the GNU extensions are Good Actually and the fact that POSIX doesn’t have them shows more like how obsolete POSIX has become and should probably be revised (and we know that this won’t ever happen). Try retrieving the exist status of a command in a pipe? Nope, POSIX sh can’t do it because nobody thought of it. And this is just one example.

                            Every time I have to use BSD userland it feels like pulling my teeth because for some reason this or that useful GNU option is missing and now I have to use some completely unreadable and terrible sed or awk invocation that’s probably also slower than what I wanted to do. This makes shell scripting even worse than it already is, and it is already painful and error-prone enough, even when using bashisms.

                            1. 8

                              Every time I have to use BSD userland it feels like pulling my teeth because for some reason this or that useful GNU option is missing and now I have to use some completely unreadable and terrible sed or awk invocation that’s probably also slower than what I wanted to do.

                              “BSD tools” is one of those terms that’s somewhat unfortunate, because there really isn’t any such thing. I’ve used FreeBSD and GNU extensively, and in my experience they’re roughly parallel in terms of feature set. Some things are a bit better in one, and some things a bit better in the other. OpenBSD, on the other hand, is much more minimal. It’s been too long that I last used NetBSD and I don’t remember. People also use “BSD tools” to mean “macOS”, but you have to remember they were imported from FreeBSD 20 years ago have since received almost no updates since then (some bugfixes and the like, but they’re certainly not “tracking upstream”).

                              All of these are “BSD tools”, but they’re all quite different.

                              1. 3

                                People also use “BSD tools” to mean “macOS”, but you have to remember they were imported from FreeBSD 20 years ago have since received almost no updates since then (some bugfixes and the like, but they’re certainly not “tracking upstream”).

                                That’s not quite true. For example, the macOS versions of the FreeBSD tools generally provide a consistent interface for getting either base-10 or base-2 units. I miss this a few times in FreeBSD where I have to try to figure out whether things want / provide MB or MiBs.

                              2. 6

                                I have the opposite experience. It wasn’t until quite recently that GNU du got the -d short option, for example, and 99% of my du invocations use -d, which was spelled --max-depth on GNU platforms. Most C/C++ code that I write works out of the box on FreeBSD / macOS and then needs me to fight glibc’s feature check macros, because BSD libcs default to exporting all features unless you opt into a specific standards-compliant mode, whereas glibc exports only a core set unless you explicitly opt into extensions (which means that there are some sets of features that you can’t have in the same compilation unit with glibc - I’ve had to split things into two .c files to get them to work with glibc in the past).

                                1. 4

                                  Oh gosh, I ran in to the du -d issue with GNU so many times. It’s been quite a while since GNU’s had it though, at least a year or five I think.

                              3. 6

                                The author is ignoring history. The libc(7) man page explains the history of Linux’s native C standard library:

                                In the early to mid 1990s, there was for a while Linux libc, a fork of glibc 1.x created by Linux developers who felt that glibc development at the time was not sufficing for the needs of Linux. … [B]y the time glibc 2.0 was released (in 1997), it was clearly superior to Linux libc, and all major Linux distributions that had been using Linux libc soon switched back to glibc.

                                This makes the three-paragraph sequence which ignores this history appear quite disingenuous.

                                I’m explaining this because we all know the history: GNU has, to this day, failed to produce a viable kernel. Linux, on the other hand, didn’t have a UNIX userland, so the only free option available was used as the natural option for the so infamous “Boot-Root” disks: Linux and the GNU userland.

                                I could just as easily say that Linux failed to produce a viable userland. It would be a grave distortion of the responsibilities of the community, though; Linux never was obligated to provide a userland, and GNU was not obligated to provide a kernel.

                                Why not use the BSD userland? Isn’t it like under a permissive license? And all BSDs predate GNU, so why didn’t that happen? I don’t have an answer for this.

                                BSDs keep their C standard libraries in sync with their kernels, as opposed to the Linux philosophy of maintaining a stable syscall interface. It makes sense to talk about freely choosing from several different C standard libraries when using Linux, but not a BSD.

                                1. 7

                                  Linux never was obligated to provide a userland, and GNU was not obligated to provide a kernel.

                                  I think the difference is that the Linux kernel project never attempted to produce a userland (though several external projects have managed it), but the GNU project did intend to develop a kernel. GNU HURD is now more-or-less usable but it took a long time. I think it’s still based on GNU Mach, though they’ve tried to port it to L4 and Coyotos.

                                  BSDs keep their C standard libraries in sync with their kernels, as opposed to the Linux philosophy of maintaining a stable syscall interface. It makes sense to talk about freely choosing from several different C standard libraries when using Linux, but not a BSD.

                                  I’m not sure about the other BSDs, but FreeBSD provides a stable syscall interface as well and allows static linking of libc (which requires a stable syscall interface). There have been other libc implementations for FreeBSD, though I don’t think any have achieved much traction.

                                2. 5

                                  I’ve been building a tool that resolves external commands in bash/shell scripts. My motive is to improve shell packaging in Nix, because I think Nix has a lot of potential to be a ~comprehensive package-manager for shell. A lot of this potential is about being able to specify what package utilities come from, rather than cope with whatever is (or isn’t) on PATH.

                                  I don’t recall seeing it expressed in these terms, but I’ve been coming to a realization like: I’m noticing a lot of wind-shaped trees, all deformed by the vacuum where the missing comprehensive package manager for shell could be.

                                  Some of these trees are people with Strong Opinions about/for/against things like POSIX and extensions, portability, shells, Shell, userland utilities, and so on. Some of these are real-world shell programs, which can be extensively deformed by having to negotiate this reality to work for a living.

                                  1. 2

                                    I’m noticing a lot of wind-shaped trees, all deformed by the vacuum where the missing comprehensive package manager for shell could be.

                                    I don’t have an opinion about shell package management, but that is a great metaphor.

                                  2. 4

                                    Fun fact: it was RMS who coined the name POSIX in the first place.

                                    https://stallman.org/articles/posix.html

                                    1. 3

                                      Really enjoyed this article:

                                      The thing in the end is that the GNU project and the FSF strike to me as very cultish and immensely insecure. Hey, guys, if you’re still reading me over there in Massachussetts… You did a great job and were a key actor in us getting what we’ve got today… But know your place and write good software. And don’t give unsuspecting newcomers such a biased version of the story that only serves your interests… mmokay?

                                      I mean, they ARE! They are, or at least were since I’ve lost touch over the last couple years, a cult of personality around RMS and the myth that the GPL is the ONLY way to actually protect free software from the evil seed Capitalism!

                                      There’s actually some value I see in having the GPL around, but I am 1000% with the author in that the FLOSS community should be looking at ways to move away from GNU software and towards efforts with more permissive licenses and less problematic leadership.

                                      1. 10

                                        I mean, if you put words in the mouth of everyone who likes the GPL, then sure, it’s a cult of personality. I think a lot of people like GPL because of its strong copyleft, which till date seems to be the best way to keep OSS projects relatively free of corporate parasitism. We don’t actually care about FSF/RMS, they did their jobs, now we can carry on with Free Software.

                                        1. 4

                                          To be fair, feoh described the FSF as a cult of personality, not “all GPL fans”.

                                          They didn’t even criticise the GPL or suggest we should stop using it (just “GNU software” - to which I agree, largely because it was almost all written in a cooperative-computing environment where malicious actors either did not exist or were the owners of the computer).

                                        2. 5

                                          There’s actually some value I see in having the GPL around, but I am 1000% with the author in that the FLOSS community should be looking at ways to move away from GNU software and towards efforts with more permissive licenses and less problematic leadership.

                                          Given how only in the last years we had drama with big companies eating smaller companies lunch and the smaller company relicensing their code to some proprietary license, on the contrary, we should probably switch to licenses like AGPL to alleviate the situation. Massive companies profit from free work in permissive licenses and don’t have to give anything back and that’s just a distorted power relationship that’s getting more and more obvious.

                                          1. 2

                                            Still waiting for the not-to-be-used-in-closed-source-training-data clause.

                                            1. 2

                                              Just my $.02 and full disclosure I’m an Amazon employee but don’t represent them in this post…

                                              I think the lesson here is that authors need to think VERY carefully about the licenses they use. The AGPL seems like a good solution if they want to avoid large companies adding value to the overall ecosystem in ways they see as hurting their bottom line.

                                              There’s something to be said for ultra-permissive licensing however, because that drives adoption in use cases that would be otherwise precluded, so it’s all about making good choices I think.

                                              1. 3

                                                The AGPL seems like a good solution if they want to avoid large companies adding value to the overall ecosystem in ways they see as hurting their bottom line.

                                                this is a hilariously uncharitable way to describe the AGPL. “know your place and write good software” is also a comical thing to say to free authors. they aren’t value machines that you can milk to appease $LARGE_EMPLOYER, they’re people with thoughts and feelings and maybe even political opinions, that they’re allowed to express.

                                                1. 1

                                                  Hi. IANAL. I know very little about open source licensing and was replying to @yawaramin ’s post where he cited that as a desirable alternative.

                                                  And I agree free software authors shouldn’t be “milked” but corporations are a-moral entities whose role is to generate profit for shareholders. The topic under discussion is how to protect authors from being treated in ways they see as undesirable. I’m not sure how to place your comment within the original context.

                                                  1. 2

                                                    right, so if your goal is not to be treated in an undesirable way by an entity that lacks morals by construction, wouldn’t you want a more restrictive license? not seeing where the AGPL/GNU fit in as wrongdoers here

                                                    1. 1

                                                      They’re not at all! AGPL is a somewhat restrictive license which some people might not prefer because of the ways it restricts the use of the code.

                                                      1. 2

                                                        ah, sorry. I read your original post as hostile towards people who prefer the AGPL over other licenses, totally my fault.