Threads for steveno

  1. 23

    The thing is that systemd is not just an init system, given it wants to cover a lot of areas and “seeps” into the userspace. There is understandably a big concern about this and not just one of political nature. Many have seen the problems the pulseaudio-monoculture has brought, which is a comparable case. This goes without saying that ALSA has its problems, but pulseaudio is very bloated and other programs do a much better job (sndio, pipewire (!)) that now have a lot of problems to gain more traction (and even outright have to camouflage as libpulse.so).

    Runit, sinit, etc. have shown that you can rethink an init system without turning it into a monoculture.

    1. 4

      In theory, having all (or at least most) Linux distros on a single audio subsystem seems like a good idea. Bugs should get fixed faster, compatibility should be better, it should be easier for developers to target the platform. But I also see a lot of negativity toward PulseAudio and people seem to feel “stuck” with it now.

      So where’s the line between undesirable monoculture and undesirable fragmentation?

      1. 21

        The Linux ecosystem is happy with some monocultures, the most obvious one is the Linux kernel. Debian has dropped support for other kernels entirely, most other distros never tried. Similarly, with a few exceptions such as Alpine, most are happy with the GNU libc and coreutils. The important thing is quality and long-term maintenance. PulseAudio was worse than some of the alternatives but was pushed on the ecosystem because Poettering’s employer wanted to control more of the stack. It’s now finally being replaced by PipeWire, which seems to be a much better design and implementation. Systemd followed the same path: an overengineered design, a poor implementation (seriously, who in the 2010s, thought that writing a huge pile of new C code to run in the TCB for your system was a good idea?) and, again, pushed because Poettering’s employer wanted to control more of the ecosystem. The fact that the problems it identifies with existing service management systems are real does not mean that it is a good solution, yet all technical criticism is overridden and discounted as coming from ‘haters’.

        1. 5

          seriously, who in the 2010s, thought that writing a huge pile of new C code to run in the TCB for your system was a good idea?

          I really want to agree with you here, but looking back at 2010 what other choice did he realistically have? Now its easy, everyone will just shout rust, but according to Wikipedia, rust didn’t have its first release till June while systemd had its first release in March.

          There were obviously other languages that were much safer than C/C++ around then but I can’t think of any that people would have been okay with. If he had picked D, for example, people would have flipped over the garbage collection. Using a language like python probably wasn’t a realistic option either. C was, and still is, ubiquitous just like he wanted systemd to be.

          1. 3

            I really want to agree with you here, but looking back at 2010 what other choice did he realistically have?

            C++11 was a year away (though was mostly supported by clang and gcc in 2010), but honestly my choice for something like this would be 90% Lua, 10% modern C++. Use C++ to provide some useful abstractions over OS functionality (process creation, monitoring) and write everything else in Lua. Nothing in something like systemd is even remotely performance critical and so there’s no reason that it can’t be written in a fully garbage collected language. Lua coroutines are a great abstraction for writing a service monitor.

            Rust wouldn’t even be on my radar for something like this. It’s a mixture of things that can’t be written in safe Rust (so C++ is a better option because the static analysis tools are better than they are for the unsafe dialect of Rust) and all of the bits that can could be written more easily in a GC’d language (and don’t need the performance of a systems language). I might have been tempted to use DukTape’s JavaScript interpreter instead of Lua but I’d have picked an interpreted, easily embedded, GC’d language (quickjs might be a better option than DukTape now but it wasn’t around back then).

            C was, and still is, ubiquitous just like he wanted systemd to be.

            Something tied aggressively to a single kernel and libc implementation (the maintainers won’t even accept patches for musl on Linux, let alone other operating systems) is a long way away from being ubiquitous.

          2. 4

            In what optics are Pipewire any kind of improvement on the situation? It’s >gstreamer< being re-written by, checking notes, the same gstreamer developers - with the sole improvements over the previous design being the use of dma-buf as a primitive, with the same problems we have with dma-buf being worse than (at least) its IOS and Android counterparts. Poettering’s employers are the same as Wim Taymans. It is still vastly inferior to what DirectShow had with GraphEdit.

          3. 14

            I’ve been using Linux sound since the bad old days of selecting IRQs with dipswitches. Anyone who says things are worse under PulseAudio is hilariously wrong. Sound today is so much better on Linux. It was a bumpy transition, but that was more than a decade ago. Let it go.

            1. 6

              Sound today is so much better on Linux.

              Mostly because of improvements to ALSA despite pulseaudio, not because of it.

              1. 4

                Yep! Pulseaudio routinely forgot my sound card existed and made arbitrary un-requested changes to my volume. Uninstalling it was the single best choice I’ve made with the software on my laptop in the last half decade.

            2. -2

              It’s no accident that PulseAudio and SystemD have the same vector, Poettering.

              1. 16

                The word you’re looking for is “developer”, or “creator”. More friendlysock experiment, less name-calling, please :)

                1. 3

                  Was Poettering not largely responsible for the virulent spread of those technologies? If so, I think he qualifies as a vector. I stand by my original wording.

                  1. 6

                    It’s definitely an interesting word choice. To quote Merriam-Webster: vector (noun), \ˈvek-tər,

                    1. […]
                      1. an organism (such as an insect) that transmits a pathogen from one organism or source to another
                      2. […]
                    2. an agent (such as a plasmid or virus) that contains or carries modified genetic material (such as recombinant DNA) and can be used to introduce exogenous genes into the genome of an organism

                    To be frank, I mostly see RedHat’s power hunger at fault here. Mr. Poettering was merely an employee whose projects, who without doubt follow a certain ideology, fit into this monopolistic endeavour. No one is to blame for promoting their own projects, though, and many distributions quickly followed suit in adopting the RedHat technologies which we are now more or less stuck with.

                    Maybe we can settle on RedHat being the vector for this, because without their publicitly no one would’ve probably picked up any of Poettering’s projects in the large scale. To give just one argument for this, consider the fact that PulseAudio’s addition to Fedora (which is heavily funded by RedHat) at the end of 2007 coincides with Poettering’s latest-assumed start of employment at RedHat in 2008 (probably earlier), while Pulseaudio wasn’t given much attention beforehand.

                    Let’s not attack the person but discuss the idea though. We don’t need a strawman to deconstruct systemd/pulseaudio/avahi/etc., because they already offer way more than enough attack surface themselves. :)

                    1. 5

                      Let’s not attack the person but discuss the idea though. We don’t need a strawman to deconstruct systemd/pulseaudio/avahi/etc., because they already offer way more than enough attack surface themselves. :)

                      This is why this topic shouldn’t be discussed on this site.

          1. 1

            I had really high hope for btrfs when it first came out. It really seems to have fizzled since. I’m guessing Facebook and, presumably Oracle, still make great use of it, though, I’m curious how exactly.

            1. 2

              I really want to use Guix, it checks all my boxes except the one where it works “out of the box” on my Intel NUC. :(

              1. 1

                Strange, why doesn’t it work on your NUC?

                1. 1

                  This blog post goes into a lot of detail about how he installed guix on his NUC, but the short answer is there are no acceptable free drivers for the wireless card.

                  https://willschenk.com/articles/2019/installing_guix_on_nuc/

              1. 2

                How does ninja shave a minute off of the build time? What can it do that make can’t?

                1. 4

                  I think part of this is down to the way that CMake uses them. The generated Ninja files rely entirely on Ninja for the build. The generated Makefiles invoke CMake bits to print useful messages. Ninja does the right thing for output by default: it buffers the output from every command and then prints it atomically for any command that produces output and shows the build command only for build steps that fail (unless you pass -v). With make, the build steps all inherit the controlling TTY and so are interleaved. It’s been years since I used make with CMake[1], but as I recall it wraps each compile command in a CMake invocation that captures the output and then tries to write it atomically. The CMake build rules also do more to produce pretty output in the form that is the default for Ninja.

                  In addition, it’s not about what Ninja can do that Make doesn’t, it’s about what Ninja doesn’t do. Ninja is not intended as a general-purpose scripting language. It doesn’t have macros, variables that are evaluated by running shell scripts, and so on, it is just a declarative description of build dependencies. It is designed with speed as its only goal, delegating all of the input complexity to a pre-build tool. Make is a far more flexible tool (particularly a modern Make such as GNU Make or BMake) that is designed to be useable without a separate configuration step, even if most uses do add one.

                  This makes it simpler to parse and get to the build step. For example, Ninja doesn’t have suffix rules. CMake is responsible for providing the full set of flags and build commands for every individual rules. This is I think Ninja is a little bit more clever about arranging the dependency tree to maximise parallelism. Not relevant for this, but important on more resource-constrained systems: Ninja is also able to have different-sized pools for different jobs, so if link steps take more memory then it can reduce the degree of parallelism during link steps.

                  [1] I have this in a file sourced from my .bashrc so CMake automatically uses Ninja:

                  # Set the default generator to Ninja.
                  export CMAKE_GENERATOR=Ninja
                  # Always generate a compile-commands JSON file:
                  export CMAKE_EXPORT_COMPILE_COMMANDS=true
                  # Give Ninja a more verbose progress indicator
                  export NINJA_STATUS="%p [%f:%s/%t] %o/s, %es "
                  
                  1. 3

                    Ninja’s scheduling is really very impressive.

                    Many years ago, I patched Jam to compile faster. My approach was different: When I had a choice of which job to start, I’d prefer the one that depended on the most recent source file. This produced amazingly fast compile times if there were errors. Builds that failed would generally fail very quickly, often in the first second. You can imagine what that does to digressions, and I thought it achieved that without any cost to successful builds.

                    Ninja was the first build tool to prove me wrong. The order in which it starts jobs is better that mine in some/many cases. Drat.

                1. 1

                  I wonder why this thread is tagged “programming” when half the posts here are not about programming or technology at all. I think the tag should be removed or at least changed to reflect the contents of the thread better.

                  1. 2

                    Looking at the comments programming seems to feature in a lot of weekend plans .

                    1. 1

                      When I posted this comment, there was one comment about sailing, another about reading and two programming related ones (well one about selling a company, kind of related to programming).

                    2. 1

                      The programming tag is defined for use when “every tag or no specific tag applies”; so it fits well here.

                      1. 1

                        But the ‘ask’ tag applies. Therefore, your argument is invalid.

                        1. 2

                          Pump your breaks boo. We’re all friends here.

                          1. 0

                            Please refrain from name calling, I was asking a genuine question and recieved no good answer. This was a civil discussion until this comment.

                    1. 32

                      Wow.

                      This article starts off with a thoughtful and nuanced exploration of some of the hard issues around open source and big tech, and then kinda jumps the shark as far as I’m concerned:

                      Work at a megacorp and either you rot working on menial things, or learn to play the politics game to get a shot at working on the one interesting project… until it gets canceled when the CEO decides to “refocus”, or when Monday comes if you work at Google.

                      Honestly, there’s no reason for this kind of attitude. You don’t want to work for big tech? Great! There’s a lot of merit for staying small and feisty and having total control. Rock on with your bad self! But don’t tar every employee working for big tech with the same brush, because:

                      1. Your preferences are not their preferences
                      2. Your goals may not be their goals
                      3. What brings you pleasure and fires your passion may not do the same for them.

                      And who are you to tell me what I do or don’t love, or where I can or can’t find fulfillment?

                      Am I asking everyone to sit around the campfire and have a sing along? Heck no! Big corporate jobs aren’t for everyone, What I would like is a bit more nuance, understanding and open mindedness and bit less vitriol.

                      The “software you can love” idea is interesting, but I guess I don’t really understand the advantage of further muddying the waters in an already super complicated licnesing landscape.

                      1. 19

                        Thanks both of you for clearing up my own concerns about the lines about Big Tech being tone deaf.

                        The “software you can love” idea is interesting, but I guess I don’t really understand the advantage of further muddying the waters in an already super complicated licnesing landscape.

                        I, personally, loved this framing. Among many issues with the Free Software movement, one I see constantly is an insistence that personal feelings of delight and good UX are sidelined in the name of “freedom”. Hate how config files feel and there’s no Settings GUI, so you’re not interested in using the software? Well, you’re an enemy of freedom. You didn’t reach the patch notes, the new software broke your system, and you’re annoyed? Well freedom is a choice, and you just chose against it. Not only does this cause people to not take free software seriously, it just means that people won’t actually use free software and will use proprietary alternatives instead, because you’re not taking the users’ desires into account when you’re designing the software.

                        I’d love to see an increased focus in free software on software that actually brings joy to its users; software that is fun to use and also free, not free but sometimes fun to use. I love the framing of that in this article. Software, at the end of the day, should solve a need.

                        1. 1

                          this feels wrong to me. sure, there are a lot of elitist people in the crowd something like the free software movement attracts, but there are also at least a handful of prominent gnu projects with ease of use as a high priority. gnome is probably the best example here, but there’s also lmms and emacs (emacs not so much, but they’ve made some efforts for semi-gui configuration and so on).

                          which is not to say that i disagree with your main point. i love gnome for how little i need to configure it (these days i turn animation off and compose key on, and that’s it), and that’s great. but i also love emacs, and that’s for very different reasons, and maybe a lot of people who love the former wouldn’t love the latter or something, but emacs has very much taken its users desires into account in its design (even if currently maintenance isn’t doing such a good job at that).

                          1. 1

                            I don’t think we disagree on anything discussed actually. Users of different systems have different expectations and needs. A user of Emacs probably has different expectations than a user of MS Word, for example.

                            there are also at least a handful of prominent gnu projects with ease of use as a high priority

                            Gnome is actually probably the only one I know. And even then, Gnome gets so much flak from many modern FOSS fans.

                            there are a lot of elitist people in the crowd something like the free software movement attracts

                            This is a question I’d love to unpack. Nothing about free software ostensibly requires elitism, but it’s present at many points of the conversation. Why?

                            1. 2

                              i think you’re right, and i misinterpreted some of what you were getting at.

                              Gnome is actually probably the only one I know. And even then, Gnome gets so much flak from many modern FOSS fans.

                              yeah, this sucks. i think a lot of that flak plays into the elitism thing, particularly as gnome seems to take a lot of inspiration from the mac aesthetic. also it appears that gnome do not consider themselves part of the gnu project, so i guess my point falls down anyway.

                              Nothing about free software ostensibly requires elitism, but it’s present at many points of the conversation. Why?

                              i find this interesting too, and i briefly touched on it in my post about nano being pretty good. fundamentally, i’d say it comes down to the fact that free software is an ethical stance. once the ethics of anything get involved, it becomes easy to consider those who don’t subscribe to, or understand, the ethical position as worse in some way, ergo, elitism. with that foundation, all sorts of other elitism can grow, and technical aptitude is an easy target in a technical environment.

                              1. 1

                                it becomes easy to consider those who don’t subscribe to, or understand, the ethical position as worse in some way, ergo, elitism

                                Yeah that might be it. It’s just unfortunate since elitism and ethics just mean that you can bully folks into accepting your UX of choice in the name of ethics, even though the ethics say nothing about UX.

                        2. 21

                          And who are you to tell me what I do or don’t love, or where I can or can’t find fulfillment?

                          Did you miss this part?

                          Big tech has been increasingly unable to create software you can love, and that’s the ability you gain by refusing to work for the silicon valley. It’s not for everyone and I can understand if somebody prefers the security of a stable job in a company too big to fail, where they can just go home after work, forget about software, and live a very comfortable life but big tech jobs are not the absolute best choice for those who really love software craftsmanship and, as it turns out, some of these people have taken a liking to Zig.

                          Also

                          The “software you can love” idea is interesting, but I guess I don’t really understand the advantage of further muddying the waters in an already super complicated licnesing landscape.

                          This has almost nothing to do with licenses, where did you get that idea? It seems to me you need to re-read the post.

                          1. 18

                            I didn’t miss that part. That part is just as tone deaf as the rest of this section of the article.

                            Would it surprise you to learn that there are a number of AWS employees who are fans of Zig?

                            People take jobs for all kinds of reasons. Those reasons can and often do have nothing at all to do with what those people may or may not love doing on their own time.

                            1. 30

                              Just like some work for AWS, I worked for Redis Labs. I don’t regret it, I learned a lot of things, and it was fun until the last phase. It also paid well and allowed me to live a good life.

                              I have no ill will towards people that work at AWS. I do have ill will towards the company, just like I have ill will towards Redis Labs given what’s happening to Redis.

                              If I had no other alternative, I would be still working at Redis Labs. I was lucky and so now I’m in a position where I can share what I think without fearing losing my job. I don’t expect anybody to become a human sacrifice in the name of software.

                              I really encourage you to read the article without seeing it as a personal attack towards you, or as a statement on licensing. It’s neither. It’s about the current dominant model of producing software, its limitations, and how the ZSF knows how to deal with big players that have different interests that conflict with ours.

                              1. 14

                                I really encourage you to read the article without seeing it as a personal attack towards you, or as a statement on licensing. It’s neither. It’s about the current dominant model of producing software, its limitations, and how the ZSF knows how to deal with big players that have different interests that conflict with ours.

                                Thanks very much for this. I will definitely do so.

                            2. 6

                              … but big tech jobs are not the absolute best choice for those who really love software craftsmanship and, as it turns out, some of these people have taken a liking to Zig.

                              It’s bullshit to say I don’t “love software craftsmanship” because I don’t work for, and frankly don’t want to work for, a non-profit startup building a new language. I will decide the “absolute best choice” for me. I really like the ideas and motivations of zig, and Andrew seems like a great guy, but your statement is just categorically false.

                              1. 5

                                I might have messed up the English, but I think you have interpreted the sentence in the opposite way of its intended meaning. The sentence doesn’t say that if you love software craftmanship big tech is the worst choice, nor that if you work in big tech you can’t love software craftmanship.

                                What the sentence hints at is the sentiment where if you are a skilled engineer that care about the craft then you have to aim for a big tech job. I used to have relatives tell me that “oh, you’re so passionate, one day you’ll work at google for sure”.

                                I guess it’s hard to have a honest conversation about something that people consider part of their identity. I personally don’t consider my job part of it. I didn’t before working for a SV startup, nor during, and neither I do now.

                                1. 1

                                  Maybe this exchange can be of interest to you.

                                  https://news.ycombinator.com/item?id=26789129

                            3. 14

                              The “software you can love” idea is interesting, but I guess I don’t really understand the advantage of further muddying the waters in an already super complicated licnesing landscape.

                              I had the exact opposite reaction: “software you can love” is nice but honestly doesn’t actually address the root cause of the problem: the profit motive is toxic to software and users. You can talk about love all you want, but the failure of Open Source shows that any approach that isn’t rooted firmly in anticapitalist principles isn’t going to be very effective in the long run.

                              Getting very strong “you don’t hate mondays; you hate capitalism” vibes from the second half of this piece.

                              1. 5

                                I had the exact opposite reaction: “software you can love” is nice but honestly doesn’t actually address the root cause of the problem: the profit motive is toxic to software and users. You can talk about love all you want, but the failure of Open Source shows that any approach that isn’t rooted firmly in anticapitalist principles isn’t going to be very effective in the long run.

                                I suppose I should remind myself that we can hold two ideas in our minds at once. Capitalism though awful has produced a LOT of incredibly good (and incredibly bad :) ) software through the years. That doesn’t necessarily contradict your statement about profit motive’s toxic effect on the process.

                                1. 4

                                  Don’t mix up co-occurrence with causality.

                                2. 4

                                  I think small shops such as Panic (https://panic.com/) are doing great: it is for profit, but not greedy. Mega corporations are in a totally different league.

                                  1. 1

                                    I would maintain that it is not capitalism specifically that is the problem, although the profit motive is usually toxic to a degree. Modern capitalism is nothing like what Adam Smith originally proposed and should possibly not even be called capitalism, I prefer the term Neo-feudalism but that is a bit tongue in cheek. I feel like large corporations have become creatures of their own which we no longer control as a species.

                                    The pattern of what is happening with humans vs. corporations is starkly reminiscent of what you see in nature when a newly introduced species of plants or animals crowd out endemic species.

                                    Should we be concerned that in the long term corporations will replace humans altogether? If so, should we not be waging all out war against them while we still can?

                                  2. 5

                                    Honestly, there’s no reason for this kind of attitude.

                                    100% agree

                                    I work for a large corporation. I’m very happy with it. I don’t see any reason to shit on me for that.

                                  1. 1

                                    Nice post and a nice read, but I’ll admit I don’t understand the fascination with re-writing everything. Its very common, and commonly not a problem, for a programming language’s kernel or core or whatever they call it, to not be written in that language. Common Lisp is the first that comes to mind with its kernel written in C. Julia has a large amount of C++ and some scheme thrown in.

                                    Will rewriting all the C in scheme really benefit Guile?

                                    1. 6

                                      Common Lisp is the first that comes to mind with its kernel written in C

                                      How’s that? Most common lisp implementations are written entirely in common lisp.


                                      It seems logical for as much as possible of a language to be written in itself. The idea is—the reason we’re working on this language is because we like it and we think it’s a good idea to write programs in it. So what does it mean that we don’t think it’s a good idea to write its own compiler in it?

                                      1. 1

                                        I guess I should have been more specific and researched a tad more.

                                        I was referring to SBCL specifically and it’s their runtime that’s written in C. https://github.com/sbcl/sbcl/tree/master/src/runtime

                                      2. 4

                                        Will rewriting all the C in scheme really benefit Guile?

                                        I think the big win here is less about technical improvements but more just the act of lowering the bus factor and making it more accessible for contributions. Ideally you want to get to the point where anyone who uses Guile can read the source, understand what it’s doing, and contribute fixes.

                                        On the other hand I’m surprised it took this long; the reader is one of those things that is A) somewhat self-contained and upstream of the rest of the compiler and B) doing work that’s very error-prone and miserable to do in C. It just makes a lot more sense if you’re going to be doing a lot of string processing to do it in a language that actually has strings!

                                        1. 1

                                          On the other hand I’m surprised it took this long

                                          Note that Guile still maintains and will maintain C reader because of bootstrap.

                                        2. 2

                                          As the article explains, Guile currently has distinction of suspendable and non-suspendable code, and it is desirable to make more code suspendable. Rewriting C to Scheme makes code suspendable. You can make C code suspendable too, but it’s more difficult.

                                          1. 1

                                            As the article explains …

                                            I read the article. I’m not against the reasoning it lays out, but everything is a cost. For example, I seriously doubt the scheme version of all of that C is going to be as fast. Is the speed differential a worthy trade-off? I’m not arguing it is or isn’t, but I am curious if those kinds of trade-offs have been considered.

                                        1. 6

                                          What is the benefit of adding const to a function? Its surprising to see what feels like a relatively minor change make the release notes.

                                          1. 17

                                            Adding ‘const’ to a rust function makes it possible to execute the function in a “const context”. Const contexts are those which are computed at compile time. A const variable’s initialization is one example of a const context. Having more things computed at compile time can have significant effects on the run time behavior of code, effectively eliminating the run time cost of producing such a value, which mattes a lot when that code may be in a tight loop.

                                            The other thing to know is that the const feature was added to rust after much of the standard library was written. In fact the language is in the process of making more and more expressions have the ability to be const. There are even some typically heap allocated values which can be const evaluated. As those features get added the standard library is adding const to functions which can be made const.

                                            So although I don’t write the release notes and I don’t know the person who does. My guess would be that they appear there because it’s a way of tracking const’s progress as an important language that has serious runtime benefits.

                                            1. 1

                                              Hm interesting, so it’s basically constexpr from C++?

                                              I started using constexpr recently, and I noticed how it has progressed over C++11, 14, 17, 20. At first you could only use functions with a single return statement. Now they are adding certain STL algorithms, etc.

                                              That is, making more and more language features valid in constexpr contexts until the STL itself can run at compile time. And probably recasting the STL itself in terms of newer features that can run in both contexts. (Memory allocation is an issue there too, but I don’t remember the exact mechanisms)

                                              Sounds like Rust is undergoing a similar evolution?

                                              1. 7

                                                Const contexts are those which are computed at compile time.

                                                I believe it more that they can be compiled at compile time, but whether they are or aren’t is up to the optimizer. But it allows the optimizer a lot more leeway in how things work; a const function is basically pure, so a lot more reduction can be done.

                                                Hm interesting, so it’s basically constexpr from C++?

                                                Pretty much exactly that, yes. Rust’s const is currently less powerful than constexpr, but it is kind of slowly creeping up on feature parity.

                                                1. 2

                                                  I think const functions can also be used in types in ways that non const functions cannot. But I don’t remember all the details

                                          1. 20

                                            Python package maintainers rarely use semantic versioning and often break backwards compatibility in minor releases. One of several reasons that dependency management is a nightmare in Python world.

                                            1. 19

                                              I generally consider semantic versioning to be a well-intentioned falsehood. I don’t think that package vendors can have effective insight into which of their changes break compatibility when they can’t have a full bottom-up consumer graph for everyone who uses it.

                                              I don’t think that Python gets this any worse than any other language.

                                              1. 20

                                                I’ve heard this opinion expressed before… I find it to be either dangerously naive or outright dishonest. There’s a world of difference between a) the rare bug fix release or nominally-orthogonal-feature-add release that unintentionally breaks downstream code and b) intentionally changing and deprecating API’s in “minor” releases.

                                                In my view, adopting SemVer is a statement of values and intention. It communicates that you value backwards compatibility and intend to maintain it as much as is reasonably possible, and that you will only knowingly break backwards compatibility on major release increments.

                                                1. 18

                                                  In my view, adopting SemVer is a statement of values and intention. It communicates that you value backwards compatibility and intend to maintain it as much as is reasonably possible, and that you will only knowingly break backwards compatibility on major release increments.

                                                  A “statement of values and intention” carries no binding commitment. And the fact that you have to hedge with “as much as is reasonably possible” and “only knowingly break” kind of gives away what the real problem is: every change potentially alters the observable behavior of the software in a way that will break someone’s reliance on the previous behavior, and therefore the only way to truly follow SemVer is to increment major on every commit. Which is the same as declaring the version number to be meaningless, since if every change is a compatibility break, there’s no useful information to be gleaned from seeing the version number increment.

                                                  And that’s without getting into some of my own direct experience. For example, I’ve been on the Django security team for many years, and from time to time someone has found a security issue in Django that cannot be fixed in a backwards-compatible way. Thankfully fewer of those in recent years since many of them related to weird old functionality dating to Django’s days as a newspaper CMS, but they do happen. Anyway, SemVer’s answer to this is “then either don’t fix it, or do but no matter how you fix it you’ve broken SemVer and people on the internet will scream at you and tell you that you ought to be following SemVer”. Not being a fan of no-win situations, I am content that Django has never and likely never will commit to following SemVer.

                                                  1. 31

                                                    A “statement of values and intention” carries no binding commitment.

                                                    A label on a jar carries no binding commitment to the contents of the jar. I still appreciate that my salt and sugar are labelled differently.

                                                    1. 2

                                                      Selling the jar with that label on it in many countries is a binding commitment and puts you under the coverage of food safety laws, though.

                                                    2. 6

                                                      Anyway, SemVer’s answer to this is “then either don’t fix it, or do but no matter how you fix it you’ve broken SemVer and people on the internet will scream at you and tell you that you ought to be following SemVer”.

                                                      What do you mean? SemVer’s answer to “this bug can’t be fixed in a backwards-compatible way” is to increment the major version to indicate a breaking change. You probably also want to get the message across to your users by pushing a new release of the old major version which prints some noisy “this version of blah is deprecated and has security issues” messages to the logs.

                                                      It’s not perfect, I’m not saying SemVer is a silver bullet. I’m especially worried about the effects of basing automated tooling on the assumption that no package would ever push a minor or patch release with a breaking change; it seems to cause ecosystems like the NPM to be highly fragile. But when taken as a statement of intent rather than a guarantee, I think SemVer has value, and I don’t understand why you think your security issue anecdote requires breaking SemVer.

                                                      1. 7

                                                        What do you mean? SemVer’s answer to “this bug can’t be fixed in a backwards-compatible way” is to increment the major version to indicate a breaking change.

                                                        So, let’s consider Django, because I know that well (as mentioned above). Typically Django does a feature release (minor version bump) every 8 months or so, and every third one bumps the major version and completes a deprecation cycle. So right now Django 3.1 is the latest release; next will be 3.2 (every X.2 is an LTS), then 4.0.

                                                        And the support matrix consists of the most recent feature release (full bugfix and security support), the one before that (security support only), and usually one LTS (but there’s a period at the end of each where two of them overlap). The policy is that if you run on a given LTS with no deprecation warnings issued from your code, you’re good to upgrade to the next (which will be a major version bump; for example, if you’re on 2.2 LTS right now, your next LTS will be 3.2).

                                                        But… what happens when a bug is found in an LTS that can’t be fixed in a backwards-compatible way? Especially a security issue? “Support for that LTS is cut off effective immediately, everybody upgrade across a major version right now” is a non-starter, but is what you propose as the correct answer. The only option is to break SemVer and do the backwards-incompatible change as a bugfix release of the LTS. Which then leads to “why don’t you follow SemVer” complaints. Well, because following SemVer would actually be worse for users than this option is.

                                                        1. 3

                                                          But… what happens when a bug is found in an LTS that can’t be fixed in a backwards-compatible way?

                                                          Why do people run an LTS version, if not for being able to avoid worrying about it as a dependency? If you’re making incompatible changes: forget about semver, you’re breaking the LTS contract, and you may as well tell drop the LTS tag and people to run the latest.

                                                          1. 1

                                                            you may as well tell drop the LTS tag and people to run the latest

                                                            I can think of only a couple instances in the history of Django where it happened that a security issue couldn’t be fixed in a completely backwards-compatible way. Minimizing the breakage for people – by shipping the fix into supported releases – was the best available option. It’s also completely incompatible with SemVer, and is a great example of why SemVer is at best a “nice in theory, fails in practice” idea.

                                                            1. 3

                                                              Why not just tell them to upgrade? After all, your argument is essentially that stable APIs are impossible, so why bother with LTS? Every argument against semver also applies against LTS releases.

                                                              1. 3

                                                                After all, your argument is essentially that stable APIs are impossible

                                                                My argument is that absolute perfect 100% binding commitment to never causing a change to observable behavior ever under any circumstance, unless also incrementing the major version at the same time and immediately dropping support for all users of previous versions, is not practicable in the real world, but is what SemVer requires. Not committing to SemVer gives flexibility to do things like long-term support releases, and generally people have been quite happy with them and also accepting of the single-digit number of times something had to change to fix a security issue.

                                                          2. 2

                                                            “Support for that LTS is cut off effective immediately, everybody upgrade across a major version right now” is a non-starter

                                                            If it’s a non-starter then nobody should be getting the critical security patch. You’re upgrading from 2.2 to 3.0 and calling it 2.2.1 instead. That doesn’t change the fact that a breaking change happened and you didn’t bump the major version number.

                                                            You can’t issue promises like “2.2.X will have long term support” because that’s akin to knowing the future. Use a codename or something.

                                                            1. 7

                                                              It’s pretty clear you’re committed to perfect technical adherence to a rule, without really giving consideration to why the rule exists. Especially if you’re at the point of “don’t commit to supporting things, because supporting things leads to breaking SemVer”.

                                                              1. 4

                                                                They should probably use something like SemVer but with four parts, e.g. Feature.Major.Minor.Patch

                                                                • Feature version changes -> We’ve made significant changes / a new release (considered breaking)
                                                                • Major version change -> We’ve made breaking changes
                                                                • Minor version change -> Non breaking new features
                                                                • Patch version change -> Other non-breaking changes

                                                                That way 2.*.*.* could be an LTS release, which would only get bug fixes, but if there was an unavoidable breaking change to fix a bug, you’d signal this in the version by e.g. going from 2.0.5.12 to 2.1.0.0. Users will have to deal with the breaking changes required to fix the bug, but they don’t have to deal with all the other major changes which have gone into the next ‘Feature’ release, 3.*.*.*. The promise that 2.*.*.*, as an LTS, will get bug fixes is honored. The promise that the major version must change on a breaking change is also honored.

                                                                SemVer doesn’t work if you try to imbue the numbers with additional meanings that can contradict the SemVer meanings.

                                                                1. 3

                                                                  This scheme is very similar to Haskell’s Package Versioning Policy (PVP).

                                                                2. 1

                                                                  I’m saying supporting things and adhering to SemVer should be orthogonal.

                                                          3. 5

                                                            every change potentially alters the observable behavior of the software

                                                            This is trivially false. Adding a new helper function to a module, for example, will never break backwards compatibility.

                                                            In contrast, changing a function’s input or output type is always a breaking change.

                                                            By failing to even attempt to distinguish between non-breaking and breaking changes, you’re offloading work onto the package’s users.

                                                            Optimize for what should be the common case: non-breaking changes.

                                                            Edit: to expand on this, examples abound in the Python ecosystem of unnecessary and intentional breaking changes in “minor” releases. Take a look at the numpy release notes for plenty of examples.

                                                            1. 7

                                                              Python’s dynamic nature makes “adding a helper function” a potentially breaking change. What if someone was querying, say, all definitions of a module and relying on the length somehow? I know this is a bit of a stretch, but it is possible that such a change would break code. I still value semver though.

                                                              1. 3

                                                                The number of definitions in a module is not a public API. SemVer only applies to public APIs.

                                                                1. 4

                                                                  If you can access it at run-time, then someone will depend on it, and it’s a bit late to call it “not public”. Blame Python for exposing stuff like the call stack to introspection.

                                                                  1. 2

                                                                    Eh no? SemVer is very clear about this. Public API is whatever software declares it to be. Undeclared things can’t be public API, by definition.

                                                                    1. 7

                                                                      Python has no concept of public vs private. It’s all there all the time. As they say in python land, “We’re all consenting adults here”.

                                                                      I’m sure, by the way, when Hettinger coined that phrase he didn’t purposely leave out those under the age of 18. Language is hard. :P

                                                              2. 1

                                                                Adding a new helper function to a module, for example, will never break backwards compatibility.

                                                                Does this comic describe a violation of SemVer?

                                                                You seriously never know what kinds of things people might be relying on, and a mere definition of compatibility in terms of input and output types is woefully insufficient to capture the things people will expect in terms of backwards compatibility.

                                                                1. 6

                                                                  No, it does not descripbe a violation of SemVer, because spacebar heating is not a public API. SemVer is very clear about this. You are right people will still complain about backward compatibility even if you are keeping 100% correct SemVer.

                                                            2. 7

                                                              I don’t think I have a naïve view of versioning; putting on my professional hat here, I have a decade of experience dealing with a dependency modeling system that handles the versions of hundreds of thousands of interrelated software artifacts that are versioned more or less independently of each other, across dozens of programming languages and runtimes. So… some experience here.

                                                              In all of this time, I’ve seen every single kind of breaking change I could imagine beforehand, and many I could not. They occurred independent of how the vendor of the code thought of it; a vendor of a versioned library might think that their change is minor, or even just a non-impacting patch, but outside of pure README changes, it turns out that they can definitely be wrong. They certainly had good intentions to communicate the nature of the change, but that intention can run hard into reality. In the end, the only way to be sure is to pin your dependencies, all the way down, and to test assiduously. And then upgrade them frequently, intentionally, and on a cadence that you can manage.

                                                              1. 1

                                                                I don’t think I have a naïve view of versioning; putting on my professional hat here, I have a decade of experience dealing with …

                                                                Here here. My experience isn’t exactly like @offby1’s but I can vouch for the rest.

                                                              2. 6

                                                                I would agree if violations were rare. Every time I’ve tried to solve dependency issues on Python, about 75% of the packages I look into have broken semver on some level. Granted, I probably have a biased sampling technique, but I find it extremely hard to believe that it’s a rare issue.

                                                                Backwards compatibility is hard to reason about, and the skill is by no means pervasive. Even having a lot of experience looking for compatibility breaks, I still let things slip, because it can be hard to detect. One of my gripes with semver is that it doesn’t scale. It assumes that tens of thousands of open source devs with no common training program or management structure all understand what a backwards breaking change is, and how to fix it.

                                                                Testing for compatibility breaks is rare. I can’t think of any Python frameworks that help here. Nor can I think of any other languages that address this (Erlang might, but I haven’t worked with it first-hand). The most likely projects to test for compatibility between releases are those that manage data on disk or network packets. Even among those, many rely on code & design review to spot issues.

                                                                It communicates that you value backwards compatibility and intend to maintain it as much as is reasonably possible, and that you will only knowingly break backwards compatibility on major release increments.

                                                                It’s more likely that current package managers force you into semver regardless if you understand how it’s supposed to be used. The “statement of values” angle is appealing, but without much evidence. Semver is merely popular.

                                                                1. 7

                                                                  I guess this depends on a specific ecosystem? Rust projects use a lot of dependencies, all those deps use semver, and, in practice, issues rarely arise. This I think is a combination of:

                                                                  • the fact that semver is the only option in Rust
                                                                  • the combination of guideline to not commit Cargo.lock for libraries + cargo picking maximal versions by default. This way, accidental incompatibilities are quickly discovered & packages are yanked.
                                                                  • the guideline to commit Cargo.lock for binaries and otherwise final artifacts: that way folks who use Rust and who have the most of deps are shielded from incompatible updates.
                                                                  • the fact that “library” is a first-class language construct (crate) and not merely a package manager convention + associated visibility rules makes it easier to distinguish between public & private API.
                                                                  • Built-in support for writing test from the outside, as-if you are consumer of the library, which also catches semver-incompatible changes.

                                                                  This is not to say that semver issues do not happen, just that they are rare enough. I’ve worked with Rust projects with 200-500 different deps, and didn’t pensive semver breakage being a problem.

                                                                  1. 5

                                                                    I would add that the Rust type system is expressive enough that many backwards incompatible changes require type signature changes which are much more obvious than violations of some implicit contract.

                                                                2. 4

                                                                  to be either dangerously naive or outright dishonest

                                                                  This phrase gets bandied around the internet so much I’m surprised its not a meme.

                                                                  SemVer is … okay, but you make it sound like lives depend on it. There’s a lot of software running mission critical systems without using SemVer and people aren’t dying everyday because of it. I think we can calm down.

                                                              3. 3

                                                                Thats the problem of the package management being so old. Back then semantic versioning wasnt that common and it never really caught on. In my opinion the PyPA should make a push to make more packages use semantic versioning. I‘m seeing this trend already, but its too slow…

                                                              1. 5

                                                                I too hated SQL when I first started with it. It grew on me though and I think it actually gets a lot of undeserved flak.

                                                                any serious DBA must examine the query plan of their SQL statements

                                                                A lot of people who aren’t DBAs write sql. And anyone who writes sql should know how to view and read a query plan. I would also suggest that knowing the database your working with (oracle, mysql, postgres, etc.) and how to write effective and efficient queries for it is no different, IMO, than telling me that I should know how to effectively write and debug efficient python programs. Every programming language has its own quirks and ins and outs. Frankly, again IMO, sql is fairly easy comparatively. Query plans are honestly pretty easy to read. For example, compare them to using gdb.

                                                                I naively think an imperative DSL for writing queries

                                                                If you look around the net enough you’ll find various people have come up with such things. They do exist though almost no one uses them. At this point sql is very entrenched and unseating it is going to take a genuine revolution in how to do it versus a couple of steps better.

                                                                So why is SQL so successful despite its shortcoming?

                                                                A large part of this, IMO, is it’s age. sql is old. All the OGs of the database world used it. And all the upstarts used it because the OGs did.

                                                                1. 3

                                                                  A lot of people who aren’t DBAs write sql. And anyone who writes sql should know how to view and read a query plan.

                                                                  I think in most businesses these points are in conflict. The only people looking at query plans are developers and DBAs (where they still exist.) Many of the people writing SQL are in business areas. They’ll possibly be using the SQL tools that developers have, but more likely they’ll be running some tool focused on report development. Smart DBAs will setup replicas and restrict access so there is risk is reduced. It’s only when these queries get moved into a more formal setting that developers and DBAs start optimizing.

                                                                  That for me, is the crux of why SQL is successful. You have a language that flourishes is a relatively non-technical environment outside of IT, yet can be directly translated into a high performance system most of the time. Much software development involves throwing it all away when these transitions happen, but SQL (and the implication of RDBMS use) allows a relatively smooth transition. For this reason, use of anything other than a relational database should be exposed to reasonable scrutiny in a business setting. Startups, ML, search, etc. are all special areas compared to the regular businesses which is all around us. It’s often hard to appreciate that when we work in tech, like optimizing, and are enthusiastic about learning new things.

                                                                1. 11

                                                                  It’s my birthday! But I also have an exam next week, so studying mostly.

                                                                  1. 2

                                                                    Happy Birthday!

                                                                    1. 2

                                                                      Happy birthday!

                                                                      1. 1

                                                                        Happy birthday!

                                                                        1. 1

                                                                          Happy Birthday!

                                                                        1. 20

                                                                          I just have been fired, so I think I will take some time to set back and enjoy my newborn.

                                                                          1. 6

                                                                            Sorry to hear that. May better opportunities come your way

                                                                            1. 3

                                                                              That is life. I will survive and come out stronger.

                                                                            2. 3

                                                                              Congratulations!

                                                                              1. 1

                                                                                Thanks

                                                                              2. 2

                                                                                Ouch, hope that doesn’t put you in a difficult situation financially.

                                                                                1. 3

                                                                                  No, I have some savings, I am left with generous additional month of payment, so I will manage. With my skillset I think that I will find next job within a month.

                                                                                2. 2

                                                                                  Wow that’s rough; I’m sorry to hear that.

                                                                                  1. 1

                                                                                    It isn’t that bad. I am not bragging, just answering the question ;) I would say it is mild inconvenience, nothing to be worried about. With my skillset I think I will find another gig soon.

                                                                                  2. 1

                                                                                    Sorry for that, but in the other hand, congrats!

                                                                                  1. 7

                                                                                    I don’t want to 💩on the author’s writeup here, because it is a decent one. I’m using it to launch another public objection to Go Generics.

                                                                                    A lot of proposals for and write ups about Go Generics seem to miss that there’s a very large group of Go users who object to Generics, and for good reason. It’s not because this group questions the efficacy of generics in solving very specific problems very well – objectors are generally well attuned to Generics’ utility. What’s objected to is the necessity of Generics. The question that we pose is do we need generics at all? Are the problems that Generics solve so important that Generics should pervade the language?

                                                                                    From the author’s conclusion

                                                                                    I was able to solve a problem in a way that was not previously possible.

                                                                                    Being able to solve problems in new ways isn’t always valuable; it can even be counter-productive.

                                                                                    1. 24

                                                                                      Nothing is necessary except an assembler. Well, you don’t even need the assembler, you can just flip the bits yourself.

                                                                                      Go has an expressiveness gap. It has some kind of big classes of algorithms that can’t be made into libraries in a useful way. Most people advocate just rewriting basically the same code over and over forever, which is kind of crazy and error-prone. Other people advocate code-generation tools with go generate, which is totally crazy and error-prone, even with the decent AST tools in the stdlib. Generics close the gap pretty well, they’re not insanely complex, and people have had decades to get used to them. If you don’t want to use them yourself, don’t use them, but accept that there are people for whom, say, the ability to just go get a red-black tree implementation that they can use with a datatype of their own choosing, without loss of type-safety or performance, will greatly improve the usefulness of the language.

                                                                                      Plus, from a purely aesthetic standpoint, it always seemed criminal to me to have a language that has first-class functions, and lexical closure, but in which you can’t even write map because its type is inexpressible.

                                                                                      1. 9

                                                                                        Go has an expressiveness gap.

                                                                                        That’s true. You’ve identified some of the costs. Can you identify some of the benefits, too?

                                                                                        1. 12

                                                                                          Easy: not having a feature protects you from bright idiots that would misuse it.

                                                                                          Honestly though, that’s the only argument I can make against generic. And it’s not even valid, because you could say this about almost any feature. It’s a fully general counter argument: give people hammers, some will whack each other’s heads instead of hitting nails.

                                                                                          Assuming basic competency of the users and assuming they were designed from the ground up, generics have practically no downsides. They provide huge benefits at almost no marginal cost. There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                          1. 2

                                                                                            They provide huge benefits at almost no marginal cost.

                                                                                            If this huge benefit is only one in a minor part of the project, or even, in a minority of projects, then it has to be balanced and thought through.

                                                                                            Right now, I don’t know many people that work Go daily, telling me that not having generics makes their day a pain.

                                                                                            Most of them told me that it’s sometimes painful, but that’s actually pretty rare.

                                                                                            There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                            Is the burden really on them? To me the it is on the program writer.

                                                                                            1. 8

                                                                                              There’s likely a survivorship bias going on there.

                                                                                              I used Go as a programming language for my side projects for years. The thing that finally got me to give it up was the lack of generics. In writing PISC, the way I had approached it in Go ended up causing a lot of boilerplate for binding functions.

                                                                                              Go is something I’d happily write for pay, but I prefer expressiveness for my side projects now, as the amount of effort that goes into a side project is a big determining factor in how much I can do in one

                                                                                              1. 3

                                                                                                There is a sizeable up-front cost for the language designer and the compiler writer, but they were willing to pay that kind of price when they set out to build a general purpose languages, didn’t they?

                                                                                                Is the burden really on them? To me the it is on the program writer.

                                                                                                Assuming we are a collaborative species (we mostly are, with lots of exceptions), then one of our goals should be minimizing total cost. Either because we want to spend our time doing something else, or because we want to program even more stuff.

                                                                                                For a moderately popular programming language, the users will far outnumber and outproduce the maintainers of the language themselves. At the same time, the languages maintainers’ work have a disproportionate impact on everyone else. To such a ludicrous extent in fact that it might be worth spending months on a feature that would save users a few seconds per day. Like compilation speed.

                                                                                                Other stuff like generic will affect fewer users, but (i) it will affect them in a far bigger way than shaving off a few seconds of compilation time would have, and (ii) those particular users tend to be library writers, and as such they will have a significant impact on the rest of the community.

                                                                                                So yes, the burden really is on the language creators and compiler writers.


                                                                                                Note that the same reasoning applies when you write more mundane software, like a train reservation system. While there is rarely any monetary incentive to make that kind of thing not only rock solid, but fast and easy to work with, there is a moral imperative not to inflict misery upon your users.

                                                                                            2. 5

                                                                                              I haven’t used Go in anger but here are some benefits from not including generics.

                                                                                              • Generics are sometimes overused, e.g. many C++ libraries.
                                                                                              • The type system is simpler.
                                                                                              • The compiler is easier to implement and high quality error messages are easier to produce.
                                                                                              • The absence of generics encourages developers to use pre-existing data structures.
                                                                                            3. 2

                                                                                              If red-black trees and map were just built in to Go, wouldn’t that solve 90% of the problem, for all practical purposes?

                                                                                              What I really miss in Go is not generics, but something that solves the same problems as multiple dispatch and operator overloading.

                                                                                              1. 3

                                                                                                Sort of, but no. There’s too many data structures, and too many useful higher-order functions, to make them all part of the language. I was just throwing out examples, but literally just a red-black tree and map wouldn’t solve 90% of the problem. Maybe 2%. Everyone has their own needs, and Go is supposed to be a small language.

                                                                                                1. 1

                                                                                                  Data structures and higher-order functions can already be implemented in Go, though, just not by using generics as part of the language.

                                                                                            4. 15

                                                                                              Technically Go does have generics, they just aren’t exposed to the end developer, except in the form of the builtin map and array types, and are only allowed for internal developers. So in a sense, Go does need generics and they already pervade the language.

                                                                                              I don’t personally have a horse in this race and don’t work with Go, but from a language-design perspective it does seem strange to limit user-developed code in such a way. I’d be curious what your thoughts on why this discrepancy is OK and why it shouldn’t be fixed by adding generics to the language.

                                                                                              1. 14

                                                                                                I don’t personally have a horse in this race and don’t work with Go, but from a language-design perspective it does seem strange to limit user-developed code in such a way.

                                                                                                Language design is all about limiting user defined code to reasonable subsets of what can be expressed. For a trivial example, why can’t I name my variable ‘int’? (In Myrddin, as a counterexample, var int : int is perfectly legal and well defined).

                                                                                                For a less trivial example, relatively few languages guarantee tail recursion – this also limits user developed code, and requires programmers to use loops instead of tail recursion or continuation passing style.

                                                                                                Adding generics adds a lot of corner cases to the type system, and increases the complexity of the language a good deal. I know. I implemented generics, type inference, and so on in Myrddin, and I’m sympathetic to leaving generics out (or, as you say, extremely limited) to put a cap on the complexity.

                                                                                                1. 3

                                                                                                  I see only two legitimate reasons to limit a user’s capabilities:

                                                                                                  1. Removing the limitation would make the implementer’s life harder.
                                                                                                  2. Removing the limitation would allow the user to shoot themselves in the foot.

                                                                                                  Limiting tail recursion falls squarely in (1). There is no way that guaranteeing tail recursion would cause users to shoot themselves in the foot. Generics is another matter, but I strongly suspect it is more about (1) than it is about (2).

                                                                                                  Adding generics adds a lot of corner cases to the type system, and increases the complexity of the language a good deal.

                                                                                                  This particular type system, perhaps. This particular language, maybe. I don’t know Go, I’ll take your word for it. Thing is, if Go’s designers had the… common sense not to omit generics from their upcoming language, they would have made a slightly different language, with far fewer corner cases they will inevitably suffer now that they’re adding it after the fact.

                                                                                                  Besides, the complexity of a language is never a primary concern. The only complexity that matters is that of the programs written in that language. Now the complexity of a language does negatively impact the complexity of the programs that result from it, if only because language space is bigger. On the other hand, this complexity has the potential to pay for itself, and end up being a net win.

                                                                                                  Take C++ for instance. Every single feature we add to it increases the complexity of the language, to almost unbearable levels. I hate this language. Yet, some of its features definitely pay for themselves. Range for for instance, while it slightly complicates the language, makes programs that use it significantly cleaner (although only locally). That particular feature definitely pays for itself. (we could discuss other examples, but this one has the advantage of being uncontroversial.)

                                                                                                  As far as I can tell, generics tend to massively pay for themselves. Not only do they add flexibility in many cases, they often add type safety (not in C++, they don’t). See for instance this function:

                                                                                                  foo : (a -> b) -> [a] -> [b]
                                                                                                  

                                                                                                  This function has two arguments (where a and be are unknown types): a function from a to b, and a list of a. It returns a list of b. From this alone, there is a lot we can tell about this function. The core idea here is that the body of the function cannot rely on the contents of generic types. This severely constraints what it can do, including the bugs it can have.

                                                                                                  So, when we write let ys = foo f xs, here’s what we can expect before we even look at the source code:

                                                                                                  • Assuming f is of type a->b, then xs is a list of a, and the result ys is a list of b.
                                                                                                  • The elements of ys, if any, can only come from elements of xs.
                                                                                                    • And they must have gone through f.
                                                                                                    • Exactly once.
                                                                                                  • The function f itself does not affect the number or order of elements in the result ys
                                                                                                  • The elements of xs do not individually affect the number or order of elements in the result ys
                                                                                                  • The only thing that affects the number or order of elements in the result ys is the size of xs (and the code of foo, of course).

                                                                                                  This is quite unlike C++, or other template/monomorphisation approaches. Done right, generics have the opportunity to remove corner cases in practice. Any language designer deciding they’re not worth their while better have a damn good explanation. And in my opinion, the explanations offered for Go weren’t satisfactory.

                                                                                                  1. 4

                                                                                                    Complexity of a language is the primary concern!

                                                                                                    Languages are tools to express ideas, but expressiveness is a secondary concern, in the same way that the computer is the secondary audience. Humans are the primary audience of a computer program, and coherence is the primary concern to optimize for.

                                                                                                    Literary authors don’t generally invent new spoken languages because they’re dissatisfied with the expressive capability of their own. Artful literature is that which leverages the constraints of it’s language.

                                                                                                    1. 4

                                                                                                      Literary authors don’t generally invent new spoken languages because they’re dissatisfied with the expressive capability of their own. Artful literature is that which leverages the constraints of it’s language.

                                                                                                      Eh, I have to disagree here. Literary authors try to stretch and cross the boundaries the of their spoken languages all the time, specifically because they search ways to express things that where not yet expressed before. To give some uncontroversial examples, Shakespeare invented 1700 new words and Tolkien invented not one, but a couple of whole new languages.

                                                                                                      I am but a very low level amateur writer, but I can tell you: the struggle with the tool to express your ideas is as real with spoken languages as it is with programming languages. It is an approach from another direction, but the results from spoken languages turn out to be as imperfect as those from programming ones.

                                                                                                      1. 1

                                                                                                        I’d argue that constrained writing is more common, if nothing else than showing ones mastery of a shared language is more impressive than adding unknown elements.

                                                                                                        Tolkien’s Elvish languages, while impressively complete, are simply used as flavor to the main story. The entire narrative instead leans heavily on tropes and language patterns from older (proto-English) tales.

                                                                                                        1. 1

                                                                                                          Yes, you have a point. I mentioned Tolkien because that was the first writer that created a new language that I could come up with. But in the end, if you want to express an idea, then your audience must understand the language that you use, otherwise they will not get your message. So common language and tropes can help a lot.

                                                                                                          However, I think your mention of constrained writing is interesting. Because in a way, that Go does not have generics, is similar to the constraint that a sonnet must follow a particular scheme in form and content. It is perfectly possible to add generics to Go, the same way as it is very possible to slap another tercet at the end of a sonnet. Nothing is stopping you, really, Expect that then it would no longer be a sonnet. Is that a bad thing? I guess not. But still almost no-one does it.

                                                                                                          I’d say that the rules, or the constraints, are a form of communication too. If I read a sonnet, I know what to expect. If I read Go, I know what to expect. Because some things are ruled out, there can be more focus on what is expressed within the boundaries. As a reader you can still be amazed. And, the same as in Go, if what you want to express really does not fit in the rules of a sonnet, or if it is not worth the effort to try it, then you can use another form. Or another programming language.

                                                                                                        2. 1

                                                                                                          Your points don’t conflict with my points, and I agree with them.

                                                                                                        3. 2

                                                                                                          Can we agree that the goal of programming languages is to reduce costs?

                                                                                                          • Cost of writing the program.
                                                                                                          • Cost of errors that may occur.
                                                                                                          • Cost of correcting those errors.
                                                                                                          • Cost of modifying the program in the face of unanticipated new requirements.

                                                                                                          That kind of thing. Now we must ask what influences the costs. Now what about increased expressiveness?

                                                                                                          A more expressive language might be more complex (that’s bad), more error prone (that’s bad), and allow shorter programs (that’s good), or even clearer programs (that’s good). By only looking at the complexity of the language, you are ignoring many factors that often matter a whole lot more.

                                                                                                          Besides, that kind of reasoning quickly breaks down when you take it to its logical extreme. No one in their right mind would use the simplest language possible, which would be something like the Lambda Calculus, or even just the iota combinator. Good luck writing (or maintaining!) anything worth writing in those.

                                                                                                          Yes, generics makes a language more complex. No, that’s not a good enough argument. If it was, the best language would only use the iota combinator. And after working years in a number of languages (C, C++, OCaml, Ptython, Lua…), I can tell with high confidence that generics are worth their price several orders of magnitudes over.

                                                                                                          1. 2

                                                                                                            I agree with you that generics can be hugely net positive in the cost/benefit sense. But that’s a judgment that can only be made in the whole, taking into account the impact of the feature on the other dimensions of the language. And that’s true of all features.

                                                                                                            1. 1

                                                                                                              Just popping in here because I have minimal experience with go, but a decent amount of experience in languages with generics, and I’m wondering: if we set aside the implementation challenge, what are some examples of the “other dimensions” of the language which will be negatively impacted by adding generics? Are these unique to go, or general trade offs in languages with generics?

                                                                                                              To frame it in another way, maybe a naive take but I’ve been pretty surprised to see generics in go being rejected due to “complexity”. I agree that complexity ought to be weighed against utility but can we be a little more specific? Complexity of what specifically? In what way will writing, reading, compiling, running, or testing code become more complicated when my compiler supports generics. Is this complexity present even if my own code doesn’t use generics?

                                                                                                              And just a final comparison on language complexity. I remember when go was announced, the big ticket feature was its m:n threaded runtime and support for CSP-style programming. These runtimes aren’t trivial to implement, and certainly add “complexity” via segmented stacks. But the upside is the ability to ergonomically express certain kinds of computational processes that otherwise would require much more effort in a language without these primitives. Someone decided this tradeoff was worth it and I haven’t seen any popular backlash against it. This feature feels very analogous to generics in terms of tradeoffs which is why I’m so confused about the whole “complexity” take. And like, maybe another naive question, but wouldn’t generics be significantly less tricky to implement than m:n threads?

                                                                                                              1. 5

                                                                                                                It isn’t just implementation complexity of generics itself. It’s also sure to increase the complexity of source code itself, particularly in libraries. Maybe you don’t use generics in your code, but surely some library you use will use generics. In languages that have generics, I routinely come across libraries that are more difficult to understand because of their use of generics.

                                                                                                                The tricky part is that generics often provides some additional functionality that might not be plausible without it. This means the complexity isn’t just about generics itself, but rather, the designs and functionality encouraged by the very existence of generics. This also makes strict apples-to-apples comparisons difficult.

                                                                                                                At the end of the day, when I come across a library with lots of type parameters and generic interfaces, that almost always translates directly into spending more time understanding the library before I can use it, even for simple use cases. That to me is ultimately what leads me to say that “generics increases complexity.”

                                                                                                                1. 2

                                                                                                                  what are some examples of the “other dimensions” of the language which will be negatively impacted by adding generics?

                                                                                                                  From early golang blog posts I recall generics add substantial complexity to the garbage collector.

                                                                                                                  The team have always been open about their position (that generics are not an early priority, and they will only add them if they can find a design that doesn’t compromise the language in ways they care about). There have been (numerous proposals rejected)[https://github.com/golang/go/issues?page=3&q=generics++is%3Aclosed+label%3AProposal] for varied reasons.

                                                                                                                  Someone decided this tradeoff was worth it and I haven’t seen any popular backlash against it

                                                                                                                  There’s no backlash against features in new languages, because there’s nobody to do the backlash.

                                                                                                                  Go has already got a large community, and there’s no shortage of people who came to go because it was simple. For them, adding something complex to the language is frightening because they have invested substantial time in an ecosystem because of its simplicity. Time will tell whether those fears were well-founded.

                                                                                                            2. 1

                                                                                                              No, expressiveness is the only reason for languages to exist. As you say, humans are the primary audience. With enough brute force, any language can get any task done, but what we want is a language that aids the reader’s understanding. You do that by drawing attention to certain parts of the code and away from certain parts, so that the reader can follow the chain of logic that makes a given program or function tick, without getting distracted by irrelevant detail. A language that provides the range of tools to let an author achieve that kind of clarity is expressive.

                                                                                                              1. 2

                                                                                                                I think we are using “expressive” differently. Which is fair, it’s not really a well-defined term. But for me, expressiveness is basically a measure of the surface area of the language, the features and dimensions it offers to users to express different ideas, idioms, patterns, etc. Importantly, it’s also proportional to the number of things that it’s users have to learn in order to be fluent, and most of the time actually exponentially proportional, as emergent behaviors between interacting features are often non-obvious. This is a major cost of expressiveness, which IMO is systemically underestimated by PLT folks.

                                                                                                            3. 3

                                                                                                              I implemented generics. You’re trying to convince me that it’s worth implementing generics. Why?

                                                                                                              Besides, the complexity of a language is never a primary concern.

                                                                                                              I disagree. I think implementation matters.

                                                                                                          2. 2

                                                                                                            That’s an intersting observation; thanks for sharing it.

                                                                                                            they just aren’t exposed to the end developer

                                                                                                            I think this supports my point better than I’m able to. Language design is just as much about what is hidden from developers as what is exposed. That generics are hidden from end users is something I greatly appreciate about Go. So when I refer to generics, I’m referring to generics used by every day developers.

                                                                                                            I’d be curious what your thoughts on why this discrepancy is OK and why it shouldn’t be fixed by adding generics to the language.

                                                                                                            In my opinion the greatest signal that Go doesn’t need generics is the wonderfully immense corpus of code we have from the last decade – all written without generics. Much of it written with delight by developers who chose Go over other langauges for it’s pleasant simplicity and dearth of features.

                                                                                                            That is not to say that some of us offasionally could have written less code if generics were available. Particularly developers writing library or framework code that would be used by other developers. Those developers absolutely would have been aided by generics. They would have written less code; their projects may have cost less to initially develop. But for every library/framework developer there are five, ten, twenty (I can’t pretend to know) end user application developers who never had the cognitive load of genericized types foisted on them. And I think that is an advantage worth forgoing generics. I don’t think I’m particularly smart. Generics make code less readable to me. They impose immense cognitive load when you’re a new developer to a project. I think there are a lot of people like me. After years of Java and Scala development, Go to me is an absolute delight with its absence of generics.

                                                                                                            1. 6

                                                                                                              In my opinion the greatest signal that Go doesn’t need generics is the wonderfully immense corpus of code we have from the last decade

                                                                                                              I don’t have a ready example, but I’ve read that the standard library itself conspicuously jumped through hoops because of the lack of generics. I see it as a very strong sign (that’s an understatement) that the language has a dire, pervasive, need for generics. Worse, it could have been noticed even before the language went public.

                                                                                                              If you had the misfortune of working with bright incompetent architects astronauts who used generics as an opportunity to make an overly generic behemoth “just in case” instead of solving the real problem they had in front of them, well… sorry. Yet, I would hesitate to accuse the language’s semantics for the failings of its community.

                                                                                                          3. 7

                                                                                                            I don’t remember exact details, it was super long ago, but I once wanted to write an editor centered around using a nontrivial data structure (“table chain” or “string table” or whatever was the name). Also the editor had some display aspect structures (~cells of terminal). At some point I needed to be able to experiment with rapidly changing the type of the object stored both in the “cells” and “chains” of the editor (e.g. to see if adding styles etc. per character might make sense from architectural point of view). If you squint, those are both kind of “containers” for characters (haskeller would maybe say monads? dunno). I had to basically either manually change all the places where the original “character” type was used, or fall back to interface{} losing all benefits of static typing that I really needed. Notably this was long before type aliases which would have possibly allowed me to push a bit further, though it’s hard for me to recall now. But the pain and impossibility of rapid prototyping at this point was so big I didn’t see it possible to continue working on the project and abandoned it. Not sure if immediately then or some time later I realized that this is the rare moment where generics would be valuable in letting me explore designs I cannot realistically explore now.

                                                                                                            In other words, what others say: nontrivial/special-purpose “containers”. You don’t need them until you do.

                                                                                                            Until then I fully subscribed to “don’t need generics in Go” view. Since then I’m in “don’t need generics in Go; except when do”. And I had one more hobby project afterwards that I abandoned for exactly the same reason.

                                                                                                            And I am fearful and do lament that once they are introduced, we’ll probably see everyone around abusing them for a lot of unnecessary purposes, and that this will be a major change to the taste of the language. That makes me respect the fact that the Team are taking their time. But I do miss them since, and if the Team grudgingly accepts the current draft as passabke, this is such a high bar that it makes me extremely excited for what’s to come, that it will be one of the best ways how this compromise can be introduced. Given that most decisions in languages are some compromises.

                                                                                                            1. 7

                                                                                                              Yeah, Go is very much not a language for rapid prototyping. It expects you to come to the table with a design already in mind.

                                                                                                              1. 2

                                                                                                                Umm, what? Honestly not sure if you’re meaning this or being sarcastic (and if yes, don’t see the point). I prototyped quite a lot of things in Go no problem. I actually hold it as one of the preferred languages for rapid prototyping if I expect I might want to keep the result.

                                                                                                                1. 5

                                                                                                                  I’m being totally serious. Go is chock full of stuff that makes typical rapid prototyping extremely difficult. A lack of a REPL. Compiler errors on unused variables. Verbose error handling. And so on. All of these things combine to make it harder to “design on the fly”, so to speak, which is what rapid prototyping frequently means.

                                                                                                                  With that said, Go works great for prototyping in the “tracer bullet” methodology. That’s where your prototype is a complete and production quality thing, and the iteration happens at a higher level.

                                                                                                                  1. 1

                                                                                                                    Got it, thanks! This made me realize that I reach for different languages in different cases for prototyping. Not yet really sure why now. But I feel that sometimes the dynamic types of Lua make me explore faster, whereas sometimes static types of Go or Nim make me explore faster.

                                                                                                            2. 4

                                                                                                              I’m going to assume you’re arguing in good faith here, but as a lurker on the go-nuts mailing list, I’ve seen too many people say “I don’t think generics are necessary” or “I haven’t heard a good enough reason for the complexity of generics”. It’s worth pointing out the Go team has collected feedback. Ian Lance Taylor (one of the current proposal’s main authors) spends a large portion of time responding to emails/questions/objections.

                                                                                                              I read a comment from someone who was on the Kubernetes team that part of the complexity of the API (my understanding is they have a pseudo-type system inside) is based on the fact that proto-Kubernetes was written in Java and the differences between the type systems compounded with a lack of generics created lots of complexity. (NOTE I don’t remember who said this, and I am just some rando on the net, but that sounds like a decent example of the argument for generics. Yes, you can redesign everything to be more idiomatic, but sometimes there is a compelling need to do things like transfer a code base to a different language)

                                                                                                              1. 1

                                                                                                                Ouch, I was wondering why the Kubernetes API looks so painfully like Java and not like Go. TIL that’s because it was literally a dumb translation from Java. :/ As much as I’m a pro-generics-in-Go guy, I’m afraid that’s a bad case for an argument, as I strongly believe it is a really awful and unidiomatic API from Go perspective. Thus I by default suspect that if its authors had generics at their disposal, they’d still write it Java-style and not Go-style, and probably still complain that Go generics are different from Java generics (and generally that Go is not Java).

                                                                                                              2. 3

                                                                                                                I don’t know if the author’s example was a good one to demonstrate the value of generics, but a cursory look at the diff would suggest he didn’t really gain anything from it. I always thought a huge benefit of generics was it saved you 10s or even 100s of lines of code because you could write one generic function and have it work for multiple types. He ended up adding lines. Granted, the author said it was mostly from tests, but still there doesn’t seem to be any dramatic savings here.

                                                                                                                1. 3

                                                                                                                  I recommend taking more than a cursory look. The value here is very much in the new library interface. In effect, the package provides generalize channels, and before the change, that generalization meant both a complicated interface, and losing compiler-enforced type safety.

                                                                                                              1. 1

                                                                                                                They look like fingerprints to me. Neat.

                                                                                                                1. 1

                                                                                                                  For sure better than what it replaces. Good job!

                                                                                                                  1. 1

                                                                                                                    @mpereira Is this a different blog post or did you just take this out of the longer blog post so it could stand alone?

                                                                                                                    Not complaining, its a great read either way, just wanted to make sure I didn’t miss anything.

                                                                                                                    1. 1

                                                                                                                      Yeah, I decided to publish them stand alone too after some feedback.

                                                                                                                      The post opens by saying that it’s part of the longer blog post btw :)

                                                                                                                    1. 13

                                                                                                                      The license is interesting; I can’t recall ever seeing this before. Is that something you added yourself? If I search for “Interface-Protection Clause” then it just turns up this project.

                                                                                                                      It’s kind of contradictory though, because first it says:

                                                                                                                      Without limiting other conditions in the License, the grant of rights under the License will not include, and the License does not grant to you, right to Sell the Software.

                                                                                                                      But then in the MIT license:

                                                                                                                      Permission is hereby granted [..] to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software.

                                                                                                                      Wouldn’t it be clearer to just modify/write a new license instead of putting a clause on top which contradicts the next clause?

                                                                                                                      1. 6

                                                                                                                        Good to notice. Then it is neither free software nor open-source.

                                                                                                                        1. 4

                                                                                                                          It is free to use, and the source is available for everyone to see and modify.

                                                                                                                          1. 19

                                                                                                                            This license will ensure that nobody will touch this software.

                                                                                                                            1. 9

                                                                                                                              I don’t see why; it doesn’t restrict almost any use case, and I don’t see any fundamental objections, and minor issues of language (if any) are fixable. Who cares about some arbitrary list that some arbitrary organisation made decades ago?

                                                                                                                              1. 4

                                                                                                                                It essentially makes it against the license to pay anyone else to make modifications of the software, and that could include employees.

                                                                                                                                That arbitrary list is fundamental to the essence of the license, so who cares? The legal department cares.

                                                                                                                                1. 4

                                                                                                                                  It essentially makes it against the license to pay anyone else to make modifications of the software

                                                                                                                                  What makes you think so?

                                                                                                                                  1. 3

                                                                                                                                    For purposes of the foregoing, “Sell” means practicing any or all of the rights granted to you under the License to provide to third parties, for a fee or other consideration (including without limitation fees for hosting the Software), a product or service whose value depends, entirely or substantially, on the third party having access to the interface of the Software.

                                                                                                                                    So this would seem to limit any ability to retrieve monetary compensation for any of the rights later listed in the license: “including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,”

                                                                                                                                    1. 4

                                                                                                                                      “Sell” means … a product or service whose value depends, entirely or substantially, on the third party having access to the interface of the Software.

                                                                                                                                      If the Preql language itself isn’t a substantial part of your product (and in most cases, it wouldn’t be), then you’re not selling it, the clause doesn’t apply, and your license is effectively MIT.

                                                                                                                                      1. 8

                                                                                                                                        If the Preql language itself isn’t a substantial part of your product (and in most cases, it wouldn’t be)

                                                                                                                                        Imagine that you are in that position – author of such product or a manager. How could you be sure that it is „not a substantial part“?

                                                                                                                                        1. 1

                                                                                                                                          If you are not sure, then it probably is. I don’t see how it could become such, unless it’s on purpose.

                                                                                                                              2. 2

                                                                                                                                I hope you’re wrong.

                                                                                                                                1. 16

                                                                                                                                  Not a lawyer, but in many of the companies I worked for, there was a whitelist of licences. If we wanted to use something and license is on the whitelist, we were free to do so. If the license is on another list, if we build a good enough case and endure several meetings with managers, we could use it. If the license is completely new, that means involving lawyers, which almost no one bothered to do and people just went looking for other stuff or implementing things from scratch.

                                                                                                                                  1. 7

                                                                                                                                    I know you don’t create these policies and that it’s not your intention to say exactly this, but what you’re saying pretty much comes down to “gimme free labour or I don’t want it” in practice.

                                                                                                                                    1. 10

                                                                                                                                      I doubt it’s about it being free; most companies would gladly shell out money to buy nonfree software, yet still be very wary about using code under a legally untested or unclear license. It’s all about ass-covering; companies are terrified they might get sued by some patent troll or copyright holder (think stuff like Oracle suing Google over their reimplementation of the Java APIs).

                                                                                                                                      1. 6

                                                                                                                                        Yes, exactly. Most of the companies I worked for were even requiring that we pay for support (when it existed), as any downtime costs money. As you nicely put, it’s all about risk mitigation – no one want to have their reputation tarnished or involved lawyers, as it means loss of money or reputation.

                                                                                                                                        1. 5

                                                                                                                                          Yeah, but how would you actually convince them to pay you if you use a standard open source license? There’s no obligation at all to pay anything, and in practice most won’t, or throw only breadcrumbs your way.

                                                                                                                                          I’m absolutely not a money-driven person, but “sending back patches” puts just as little bread on the table as “make me free stuff for exposure” does for artists. I feel a lot of people are rather naïve about this. Sure, open source ideals and all of that are wonderful and in a perfect world we’d all be happily coding together and paying our fair share. But not everyone is playing by the same rulebook here. Jeff Bezos’ goal is to make profit, nothing more, nothing less. If he can make profit by paying you $1 instead of $10 while netting $100k then he will have no qualms in doing so. He doesn’t care one iota if that $1 is a living wage or anywhere near “fair” or proportional. Jeff Bezos is hardly an outlier here.

                                                                                                                                          How much money does CodeYellow send to Debian, PostgreSQL, PHP, Laraval, and all the other projects you use (assuming you still work there)? Perhaps CodeYellow is better most – that would be nice – but all companies I worked for that amount was literally “$0”. And sure, CodeYellow provides plenty of added value, but just as a builder pays for their raw materials, it doesn’t strike me as unreasonable that CodeYellow (and AWS, etc. etc.) also pays for them.

                                                                                                                                          At any rate, if you’re concerned about the non-standard license then you can simple contact the author to buy a different one. Sure, it’s a bit more effort, but you can’t have your cake and eat it too.

                                                                                                                                          1. 2

                                                                                                                                            Yeah, but how would you actually convince them to pay you if you use a standard open source license?

                                                                                                                                            That’s one of the great unsolved problems of our age. I think there are some nice initiatives around (like for example GitHub’s sponsoring option) but they’re not gaining a lot of traction yet.

                                                                                                                                            There’s also the question of distribution: how do you fairly distribute any payment/sponsorship across a large open source project? One-man projects are a different story of course, but a lot of the widely used projects are multi-person efforts.

                                                                                                                                            I’m absolutely not a money-driven person, but “sending back patches” puts just as little bread on the table as “make me free stuff for exposure” does for artists. I feel a lot of people are rather naïve about this.

                                                                                                                                            Totally agree on this. For myself, open source was always more about freedom of source and didn’t care as much about money, but then the vast majority of companies don’t subscribe to these ideals and just take the free software and lock it away. And it could be argued the world would be a much better place if people would be able to make a living off of free software, because it would mean more viable free software could make its way into the hands of actual users, and less developers would feel like they’d have to work on proprietary software to support their free software producing “hobby”.

                                                                                                                                            How much money does CodeYellow send to Debian, PostgreSQL, PHP, Laravel, and all the other projects you use (assuming you still work there)?

                                                                                                                                            I left a couple of months ago, I’m now working at bevuta IT on a project for the EU, which is likely to be open sourced.

                                                                                                                                            Code Yellow actually sponsored a CI test server for the CHICKEN project and of course I was paid to work on some open source projects of our own. But I agree with you, most companies don’t pay at all, and as far as I remember, Code Yellow did not sponsor any of the projects we were actually using day to day (unless you count the odd code contribution and bugfix) that weren’t developed in-house.

                                                                                                                                            1. 5

                                                                                                                                              I think GitHub sponsors is great and I don’t want to knock it, but it’s also not really a viable business model. You certainly can make a living out of it (people are), but it’s not easy, mostly works well for individual devs (rather than business), and is not suitable for all projects either (generally only for “user visible” projects, and even thn only for “dev users”, rather than the general population).

                                                                                                                                              Take something like vim-go for example; the current maintainer nets $102/month (the previous maintainer got a bit more, about $400/month IIRC). All of that is nice pocket change, but is it really in proportion to the usage? Even with just 5,000 companies using it (probably a low estimate) times $1/month we would be talking about $5k/month. We could discus what a “fair” amount would be exactly for a long time, but these are tiny investments, and if you consider your devs will get a better editor in return it’s an investment that will actually give you a return too.

                                                                                                                                              The entire world and their mother is running OpenSSH, but OpenBSD still gets in to intermittent financial troubles. They run on $300k/year (2020 fundraising goal; CAD, about 240k USD), which may sound like a lot, but this is the salary of what, one dev in silicon valley? Maybe two? Even by the lower European wages we’d be talking about ~4 devs. Spread out over all users of the software this is nothing. The kind of money you won’t even bother picking up if you drop it.

                                                                                                                                              If every company would pay $1 for ever open source tool, library, or application they use we’d be talking about a $100-$200/month investment, at the most, and often probably much smaller (let’s ignore npm and it’s 20k dependencies, that’s a special problem). This wouldn’t solve everything, but it’s a simple start.

                                                                                                                                              Yet it’s not happening; not even close. In my observation most people sponsoring projects tend to be private individuals like you and me, who care about these things. I feel a big reason it’s not happening is because companies don’t have to and just can’t be bothered. So what you need is the proverbial “stick behind the door”, which is what these kind of licenses are. It’s sad that this is needed, but it is what it is.

                                                                                                                                              Or, maybe, what we need is a single organisation that you can tell “we use these projects” and pay that $100/month (or whatever is appropriate), and they distribute it to the projects. This would certainly simplify the administration and burden for the companies, as well as lower payment processing overhead and the like. One would imagine that something like the FSF would do that, but they seem more concerned with their crusade to fight the Open Source pagan heretics, or something, and the OSI doesn’t seem to be doing much at all except bicker amongst themselves and write blogspam.

                                                                                                                                              It’s kind of hilariously sad that GitHub Sponsors is an initiative from a company mostly writing proprietary software. I’m not a purist and don’t see it as a terribly great problem as such, but my irony meter is going in to overload.

                                                                                                                                              For myself, open source was always more about freedom of source and didn’t care as much about money

                                                                                                                                              Yeah, I don’t really care about it as an end-goal either; just as a means to ensure viability.

                                                                                                                                              I left a couple of months ago, I’m now working at bevuta IT on a project for the EU, which is likely to be open sourced.

                                                                                                                                              Aha, nice; it mentions that “CHICKEN Scheme is one of our favourite programming languages” :-) Looks like Felix is working there as well?

                                                                                                                                              1. 4

                                                                                                                                                Looks like we’re in total agreement about this.

                                                                                                                                                So what you need is the proverbial “stick behind the door”, which is what these kind of licenses are. It’s sad that this is needed, but it is what it is.

                                                                                                                                                Here you run into the problem that these licenses are “new” and untested in court, so legal departments won’t touch them with a ten-foot pole. It also doesn’t help that they don’t match the OSI’s criteria (or the FSF’s), which such licenses won’t be endorsed by trusted parties.

                                                                                                                                                maybe, what we need is a single organisation that you can tell “we use these projects” and pay that $100/month (or whatever is appropriate), and they distribute it to the projects.

                                                                                                                                                This sounds like a good solution and something the FSF (or FSFe) could be doing. Have you contacted them to suggest something like this? It seems that recently the FSF is doing some good advocacy (like the public money, public code campaign and the ethical tech giving guide) instead of just preaching to the choir like they used to.

                                                                                                                                                Aha, nice; it mentions that “CHICKEN Scheme is one of our favourite programming languages” :-) Looks like Felix is working there as well?

                                                                                                                                                Yep, I have several colleagues that I’ve known for a while and am happy to finally be working with them!

                                                                                                                                                1. 1

                                                                                                                                                  While I’ve argued the that FSF/OSI should do more direct funding of projects many times before, I didn’t have that specific idea until I wrote that comment. So, no, I haven’t contacted them yet :-) You’re the first I told it to! Consider yourself special!

                                                                                                                                                  I hope to finish my article about this soon™; I’ll be sure to post it to those kind of channels as well. It’s good to hear the FSF is improving (finally…)

                                                                                                                                                  As for the license, you can still sell access to a more standard license if you want, like SQLite does (even though it’s public domain).

                                                                                                                                        2. 5

                                                                                                                                          This is exactly how corporations or software departments often work. If something is free of charge, they will use it. Why not? On the other hand, they are usually ready to pay for proprietary software or external services. So the strong copyleft + dual-licensing is the way to go. Your software will then fall in the „proprietary software“ category for them, but it is no problem. Their lawyers will be happy in the same way as if the company buys a license for Photoshop, AutoCAD, Oracle DB, Total Commander, WinRAR etc. and you will get money.

                                                                                                                                      2. 3

                                                                                                                                        Well with regular software licensed in common open source ways, nobody else is prevented from modifying it or hosting it for a fee. You have essentially limited the users of to exactly one person who can do anything involving any kind of money for the software. That is a non-starter for almost anyone outside of the plain hobbyists - and it effectively preventing any adoption besides a few insignificant users.

                                                                                                                                        Looking at the license, it isn’t entirely clear what “interface of the Software” means either.

                                                                                                                                        1. 4

                                                                                                                                          Then I think you misunderstood.

                                                                                                                                          It is possible to create commercial software that uses Preql, with or without modifications, and sell it however you want. The only limitation is that in that case, you can’t provide the language itself to your users (i.e. the interface). That restricts specific use-cases, such as offering it as a cloud service to developers, or selling a Preql plugin to existing database software.

                                                                                                                                          1. 2

                                                                                                                                            It is possible to create commercial software that uses Preql, with or without modifications, and sell it however you want.

                                                                                                                                            you can’t provide the language itself to your users (i.e. the interface).

                                                                                                                                            Can you explain what exactly that means? How do I include the language without modifications but not provide the language to the users??

                                                                                                                                            1. 6

                                                                                                                                              It means you can’t provide a product which is built around the ability for users to use the language directly.

                                                                                                                                              Integrating it in your desktop or web app and selling that is okay.

                                                                                                                                              Making AWS SuperQL™® or CloudPostMyMariaSQLiteBlockchainML++ which are essentially cloud versions of of Preql (and thus expose the language/interface to users directly, as a major feature) would not be okay.

                                                                                                                                              In short, it’s intended to prevent exploitation from these kind of people, as they have done before and will undoubtedly continue to do in the future.

                                                                                                                                              1. 3

                                                                                                                                                Let’s say you are writing a product that uses Preql to save and fetch user data from the database. It is called from internal routines, that respond to unrelated user activity. In that case, there is no reason for the user to ever write any Preql code, which means he isn’t being given access to the language. In that case, Preql can be used under MIT conditions, and you can sell your product without a worry.

                                                                                                                                      3. 3

                                                                                                                                        “free to use” and “free software” are not the same thing. I don’t know if your rider actually excludes you from being free software or not.

                                                                                                                                    2. 6

                                                                                                                                      I’ve spent a lot of time thinking about the license, and I could not find anything that fit what I wanted. It’s a long discussion, but I believe the popular licenses are flawed, in that they don’t properly support those who want to be able to write open-source for a living.

                                                                                                                                      The one that came the closest was The Commons Clause (https://commonsclause.com/), and I modified it. But the “contradiction” you mention was already there, purposely put by a well-respected lawyer.

                                                                                                                                      Wouldn’t it be clearer to just modify/write a new license

                                                                                                                                      Maybe I think too much like a programmer, but I don’t see why I shouldn’t write my license using a “library license”, well-tested and recognized, instead of reinventing the entire wheel from scratch.

                                                                                                                                      1. 23

                                                                                                                                        Maybe I think too much like a programmer, but I don’t see why I shouldn’t write my license using a “library license”, well-tested and recognized, instead of reinventing the entire wheel from scratch.

                                                                                                                                        This is akin to having a lawyer write your auth code.

                                                                                                                                        1. 3

                                                                                                                                          Right; fair enough. I had temporarily forgotten about the Commons Clause 😅 I never really liked the “tacking on” of that either, but if it works (legally speaking), it works, so whatever.

                                                                                                                                          I think your license and modifications to the Commons Clause make sense, by the way; by removing the “consulting/ support services related to the Software” and changing “whose value derives, entirely or substantially, from the functionality of the Software” to “the third party having access to the interface of the Software” it limits the scope a lot to what you actually want to prevent.

                                                                                                                                          That being said; I’m very far removed from being a lawyer. Perhaps @kemitchell wants to chime in?

                                                                                                                                          1. 3

                                                                                                                                            don’t properly support those who want to be able to write open-source for a living.

                                                                                                                                            I recommend using a strong copyleft license (GPLv3 or AGPL) and dual-licensing. You can sell licenses that allows use inside proprietary software or services (that do not provide source code and rights to the end-user).

                                                                                                                                            And if someone complies with the copyleft license and provides all the rights and code to the end-user and make some money while doing it – does it harm you in any way?

                                                                                                                                            1. 2

                                                                                                                                              What the commons clause claims in its FAQ vs what it says in its content are very different. To quote the clause itself:

                                                                                                                                              For purposes of the foregoing, “Sell” means practicing any or all of the rights granted to you under the License to provide to third parties, for a fee or other consideration (including without limitation fees for hosting or consulting/ support services related to the Software), a product or service whose value derives, entirely or substantially, from the functionality of the Software. Any license notice or attribution required by the License must also include this Commons Clause License Condition notice.

                                                                                                                                              I specifically want to call out this bit:

                                                                                                                                              a product or service whose value derives, entirely or substantially, from the functionality of the Software.

                                                                                                                                              What does “substantially” mean? When does use of software under the commons clause become substantial enough to be in violation of the license? More than anything, lawyers hate ambiguity. This line is very ambiguous.

                                                                                                                                              Your modified version of the license reads:

                                                                                                                                              a product or service whose value depends, entirely or substantially, on the third party having access to the interface

                                                                                                                                              “Access” is not defined. One could argue that “access” means that the code paths are being exercised by code available to a third party. This modification, in my opinion, is more restrictive and more ambiguous than the original commons clause.

                                                                                                                                              I understand that your objective is to prevent this from being repackaged and sold directly as a product. This is a pretty hot topic in general right now and I’d say no one has quite figured it out. That said, based on my experience with 3p OSS licensing, no company would knowingly accept this license for any use case. Commons clause is already a tough sell and the modifications made make it near impossible. If you’re open to feedback, I would strongly consider using the Commons Clause language without modification.

                                                                                                                                              1. 2

                                                                                                                                                Thanks, I appreciate the feedback. I was also concerned about the meaning of substantially, but since it only applies to access to the interface, that shouldn’t be a problem. I suppose I can be more specific about the use of the word “access”. In essence, it can be solved by being described as a dataflow query (i.e. taint analysis).

                                                                                                                                                But the Commons Clause, as it is, is too restrictive, and will drive away many of the early adopters.

                                                                                                                                                I’m definitely open to feedback. Like you said, no one has figured it out yet. And I might change my license in the future. But at the moment, I don’t see an alternative.

                                                                                                                                              2. 2

                                                                                                                                                Maybe I think too much like a programmer, but I don’t see why I shouldn’t write my license using a “library license”, well-tested and recognized, instead of reinventing the entire wheel from scratch.

                                                                                                                                                Common practice is that amendments* to a license loosens original rules and adds more rights to the licensee. So using a name of a well-known license as your „library license“ and adding a restricting amendment is at least confusing.

                                                                                                                                                Using the name of an original free software / open-source license is quite parasitic and harmful. Common users will see Apache or MIT + something else that they do not read. Yes, it is their fault that they do not read and understand it fully. But I consider it unethical to exploit someone else’s mistakes this way. Such common users will then doubt whether Apache, MIT or any other free software license grants them rights that it actually grants them.

                                                                                                                                                Nobody prevents you from creating a new license or a new category of software. But please, do not exploit existing licenses and categories, well known-names and do not confuse users.

                                                                                                                                                *) for example the Classpath Exception that could be added to GPLv2 – licensee can do everything allowed by GPLv2 + something else allowed by the amendment.

                                                                                                                                                1. 1

                                                                                                                                                  Have you thought about using something like https://spdx.org/licenses/BUSL-1.1.html ?

                                                                                                                                              1. 1

                                                                                                                                                I love systemd user services. I recently started using them for emacs. Now that I’ve realized how easy they are I’ll probably make a lot more use of them.

                                                                                                                                                1. 9

                                                                                                                                                  I use elfeed under emacs. Before that, I used rss2email.

                                                                                                                                                  1. 2

                                                                                                                                                    How are you liking it? I have used it some, but find myself more often using the self-hosted TT-RSS instance I have on a personal server.

                                                                                                                                                    1. 2

                                                                                                                                                      I have no complaints. I’ve been using it for about 4 months or so. My usage is pretty basic; I don’t follow a ton of feeds.

                                                                                                                                                      1. 2

                                                                                                                                                        I follow around 100 or so, and it does take a bit to update. I suppose there are a couple main differences I find with a web-based GUI or API:

                                                                                                                                                        • The web version has centralized state management, with a mobile app view as well.
                                                                                                                                                        • That central state is also updated in the background, rather than when I prompt it to fetch new stories.
                                                                                                                                                        • Local storage is great for quick access and search though, lighter than web round-trips.

                                                                                                                                                        A background refresh that happens periodically might be fine, except that I think it’s hard on whatever else I have going on in Emacs at the time. So for now, I kick off an update, then go to my browser or something while it runs.

                                                                                                                                                      2. 1

                                                                                                                                                        I use it too, I like being able to define feed hooks in elisp (filtering certain topics, posts and people out, tagging others, etc.). I don’t really have any complaints with it, it’s much better than struggling against gnus.

                                                                                                                                                        1. 2

                                                                                                                                                          I don’t think I’ve ever used gnus for feeds, just email and newsgroups. It was fine then for what I did, but do use mu4e now. Just doing basic things with elfeed currently, but maybe I should check into those other features to get more out of it.

                                                                                                                                                      3. 1

                                                                                                                                                        I just looked into this and I noticed the “db” it uses appears to just be a text file that’s maintained through some elisp code. Is that common in the emacs world? I looked around, it appears using something like sqlite is totally possible.

                                                                                                                                                        1. 1

                                                                                                                                                          It is, org-roam uses sqlite for its database. I don’t know how that might map to how elfeed does things though.

                                                                                                                                                          1. 1

                                                                                                                                                            Most emacs extensions that need some kind of data persistance just write the data as elisp in a text file, yeah.

                                                                                                                                                        1. 16

                                                                                                                                                          Not to be glib, but I don’t.

                                                                                                                                                          If I have personal stuff going on that requires more than a page in notebook, I probably have too much going on.

                                                                                                                                                          1. 3

                                                                                                                                                            Totally with you. I don’t either.

                                                                                                                                                            It’s not about being glib or pessimistic. For me it’s about my sanity. The list of things I want to do and haven’t will always be longer than the list of things I have. In a perfect world, I would have PhD, be the author of a wildly popular programming language or library or something, and be highly regarded in my field. There’s nothing wrong with being down to earth and realizing at this point in my life the odds of any of those happening is slim. I’m okay with that.

                                                                                                                                                            Now on a positive note, that doesn’t mean I’m not constantly out to do something I haven’t. Or trying to accomplish a new goal in my life. I just try not look back.

                                                                                                                                                            1. 1

                                                                                                                                                              For me having more than that page is a sign that I am not working on the right thing, or not in the right way. For work and other commitments it is different, but personal “goals” and projects are the only place where you can do things purely driven by intrinsic motivation. If you have enough of that, you don’t need a system to guide you, or a system that helps staying disciplined. It is very important for my mental health to have a place in my life where that can happen.

                                                                                                                                                              So, if I have a day off and I can do whatever I want, what should I work on? On whatever I feel like. It’s that simple.