Threads for singpolyma

  1.  

    In case it’s unclear from the title: Haskell does not have forall but there is a GHC extension which adds it.

    1. 30

      I think size of the program, and the team maintaining it, is an important factor in the static vs dynamic discussion.

      I’m in the “I don’t make type errors, and if I do, I can shake them out with a few tests” camp for as long as I can comprehend what’s going on in the codebase.

      But when the code grows to the point I can no longer remember where everything is, e.g. I can’t find all callers of a function, dynamism starts to become a burden. When I need to change a field of a struct, I have to find all uses of the field, because every missed one is a bug that’s going to show up later. At some point that becomes hard to grep for, and even impossible to account for in reflection-like code. It can degrade to a bug whack-a-mole, promote more and more defensive programming patterns, and eventually fear of change.

      I’ve had good experiences with gradually-typed languages. They stretch this “size limit” of a program a lot, while still allowing use of duck typing where it helps, without bringing complexity of generic type systems.

      1. 10

        “Dynamic typing falls apart with large team/large codebase” is one of those cliché arguments that doesn’t really contribute usefully, though.

        Also your presentation of it has multiple issues:

        • Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.
        • Tooling for dynamically-typed languages has come a long way in the decades since this argument was first raised. You can just get an IDE and tell it to track down and rename references for you. And if your complaint is that it’s harder/impossible to do through “reflection-like code”, well, people can write metaprogram-y reflection stuff in statically-typed languages too.
        • Ultimately, if your codebase has lots of functions or methods that are called from huge numbers of disparate places, to such a degree that you can’t safely work with it without an IDE doing full static analysis to track them all for you, that’s a code smell in any language, in any typing discipline.
        1. 18

          Static languages can verify all metaprogramming is type correct. IDE heuristics can not. In Rust you can write a macro and the compiler will expand and type check it. That kind of stuff is impossible in dynamic languages.

          1. 8

            Static languages can verify all metaprogramming is type correct.

            This is probably going to get off-topic into arguing about the exact definition of “statically-typed”, but: I think that if you venture outside of languages like Rust (which seem to deliberately limit metaprogramming features precisely to be able to provide guarantees about the subset they expose), you’lll find that several languages’ guarantees about ahead-of-time correctness checks start being relaxed when using metaprogramming, runtime code loading, and other “dynamic-style” features. Java, for example, cannot actually make guarantees as strong as you seem to want, and for this among other reasons the JVM itself is sometimes referred to as the world’s most advanced dynamically-typed language runtime.

            There also are plenty of things that seem simple but that you basically can’t do correctly in statically-typed languages without completely giving up on the type system. Truly generic JSON parsers, for example. Sure, you can parse JSON in a statically-typed language, but you either have to tightly couple your program to the specific structures you’ve planned in advance to handle (and throw runtime errors if you receive anything else), or parse into values of such ultra-generic “JSON object” types that the compiler and type system no longer are any help to you, and you’re effectively writing dynamically-typed code.

            1. 3

              Dynlangs are definitely better for data that isn’t structured as well.

              C#’s dynamic keyword feels like a perfect fit for this situation without having to give up static typing everywhere else. Hejlsberg is ahead of the curve, per usual.

              1. 3

                for this among other reasons the JVM itself is sometimes referred to as the world’s most advanced dynamically-typed language runtime

                Aren’t runtimes always “dynamically typed”? What does it mean for a runtime to be “statically typed”?

                or parse into values of such ultra-generic “JSON object” types that the compiler and type system no longer are any help to you, and you’re effectively writing dynamically-typed code.

                It sounds like you’re arguing that the worst case for static type systems is equivalent to the best case for dynamic type systems, which doesn’t seem like a ringing endorsement for dynamic type systems. That said, I don’t even think this is true for this JSON-parsing example, because you could conceive of a generic JSON parser that has different unmarshaling strategies (strict, permissive, etc). Further, as static type systems are adopted more widely, this sort of poorly-structured data becomes rarer.

                1. 5

                  Aren’t runtimes always “dynamically typed”?

                  Some more so than others. Rust, for all its complexity as a language, is mostly shoving that complexity onto the compiler in hopes of keeping the runtime relatively simple and fast, because the runtime doesn’t have to do quite as much work when it trusts that there are classes of things the compiler simply prevents in advance (the runtime still does some work, of course, just not as much, which is the point).

                  But a language like Java, with runtime code loading and code injection, runtime reflection and introspection, runtime creation of a wide variety of things, etc. etc. does not get to trust the compiler as much and has to spend some runtime cycles on type-checking to ensure no rules are being broken (and it’s not terribly hard to deliberately write Java programs that will crash with runtime type errors, if you want to).

                  That said, I don’t even think this is true for this JSON-parsing example, because you could conceive of a generic JSON parser that has different unmarshaling strategies (strict, permissive, etc).

                  If you want truly generic parsing, you’re stuck doing things that the compiler can’t really help you with. I’ve seen even people who are quite adept at Haskell give up and effectively build a little subset of the program where everything is of a single JSON type, which is close enough to being dynamically typed as makes no difference.

                  Further, as static type systems are adopted more widely, this sort of poorly-structured data becomes rarer.

                  My experience of having done backend web development across multiple decades is that poorly-structured data isn’t going away anytime soon, and any strategy which relies on wishing poorly-structured data out of existence is going to fail.

                  1. 2

                    Aren’t runtimes always “dynamically typed”?

                    If you eschew these needlessly binary categories of static vs dynamic and see everything on a scale of dynamism then I think you’ll agree that runtimes are scattered across that spectrum. Many even shift around on that spectrum over time. For example, if you look at the history of JSR 292 for adding invokedynamic to the JVM you’ll find a lot of cases where the JVM used to be a lot less dynamically typed than it is today.

                  2. 2

                    There’s no reason you can’t parse a set of known JSON fields into static members and throw the rest into an ultra-generic JSON object.

                    1. 3

                      Those are the options I said are available, yes.

                      1. 1

                        I mean, you can do both at once for the same value, getting the benefits of both.

                2. 7

                  Fail is too harsh. Unless you’re writing some rocket navigation system, a project is not going to outright fail because of software defects. Run-time type errors merely add to other bugs that you will need to fix, and I argue that bugs caused by runtime type errors are less of a problem in small programs.

                  I don’t know of any robust tooling for refactoring large JavaScript projects. Of course most languages have some type-system escape hatches, but I expect languages like JS to use hard-to-analyze type-erasing constructs much more often.

                  I disagree that having callers beyond your comprehension is automatically a code smell. It’s a natural state of things for libraries, for example. Ideally libraries should have a stable API and never change it, but it’s not always that easy, especially for internal libraries and reusable core pieces of large projects that may need to evolve with the project.

                  It’s not just about IDEs. Compilation will also track down all type errors for you, regardless of where and when these errors happen. When working with teams, it may be someone else working on some other component. In this case the types are a way to communicate and coordinate with others.

                  You can make a mess in any language, but how easy is to make a mess varies between languages. Languages that prevent more errors will resist the mess for longer.

                  1. 2

                    I expect languages like JS to use hard-to-analyze type-erasing constructs much more often.

                    Why do you expect this?

                    I disagree that having callers beyond your comprehension is automatically a code smell.

                    Even if it’s an internal library, why don’t other internal codebases have a single clear integration point with it? And why does everything else need to have lots of knowledge of the library’s structure? This definitely is a code smell to me – the Law of Demeter, at least, is being violated somewhere, and probably other design principles too.

                    Languages that prevent more errors will resist the mess for longer.

                    This is veering off into another clichéd and well-trod argument (“static typing catches/prevents more bugs”). I’ll just point out that while proponents of static typing often seem to take it as a self-evident truth, actually demonstrating its truth empirically has turned out to be, at the very least, extremely difficult. Which is to say: nobody’s managed it, despite it being such an “obvious” fact, and everybody who’s tried has run into methodological problems, or failed to prove any sort of meaningful effect size, or both.

                    1. 1

                      Why do you expect this?

                      Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                      why don’t other internal codebases have a single clear integration point with it?

                      The comprehension problems I’m talking about that appear in large programs also have a curse of being hard to explain succinctly in a comment like this. This is very context-dependent, and for every small example it’s easy to say the problem is obvious, and a fix is easy. But in larger programs these problems are harder to spot, and changes required may be bigger. Maybe the code is a mess, maybe the tech debt was justified or maybe not. Maybe there are backwards-compat constraints, interoperability with something that you can’t change, legacy codebase nobody has time to refactor. Maybe a domain-specific problem that really needs to be handled in lots of places. Maybe code is weirdly-shaped for performance reasons.

                      The closest analogy I can think of is “Where’s Waldo?” game. If I show you a small Waldo picture, you’ll say the game is super easy, and obviously he’s right here. But the same problem in a large poster format is hard.

                      1. 4

                        Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                        You are once again assuming that statically-typed languages catch/prevent more errors, which I’ve already pointed out is a perilous assumption that nobody’s actually managed to prove rigorously (and not for lack of trying).

                        Also, the explanation you give still doesn’t really make sense. Go look at some typical Python code, for example – Python’s metaprogramming features are rarely used and their use tends to be discouraged, and easily >99% of all real-world Python code is just straightforward with no fancy dynamic tricks. People don’t choose dynamic typing because they intend to do those dynamic tricks all the time. They choose dynamic typing (in part) because having that tool in the toolbox, for the cases when you need it or it’s the quickest/most straightforward way to accomplish a task, is incredibly useful.

                        The comprehension problems I’m talking about that appear in large programs also have a curse of being hard to explain succinctly in a comment like this

                        Please assume that I’ve worked on large codebases maintained by many programmers, because I have.

                        And I’ve seen how they tend to grow into balls of spaghetti with strands of coupling running everywhere. Static typing certainly doesn’t prevent that, and I stand by my assertion that it’s a code smell when something is being called from so many disparate places that you struggle to keep track of them, because it is a code smell. And there are plenty of patterns for preventing it, none of which have to do with typing discipline, and which are well-known and well-understood (most commonly, wrapping an internal interface around a library and requiring all other consumers in the codebase to go through the wrapper, so that the consuming codebase controls the interface it sees and has onlyu a single point to update if the library changes).

                        1. 2

                          I’ve worked on large codebases maintained by many programmers, because I have. And I’ve seen how they tend to grow into balls of spaghetti with strands of coupling running everywhere. Static typing certainly doesn’t prevent that . . .

                          No, definitely not, agreed. But static typing definitely improves many/most dimensions of project maintainability, compared to dynamic typing. This isn’t really a controversial claim! Static typing simply moves a class of assertions out of the domain of unit tests and into the domain of the compiler. The question is only if the cost of those changes is greater or lesser than the benefits they provide. There’s an argument to be made for projects maintained by individuals, or projects with lifetimes of O(weeks) to O(months). But once you get to code that’s maintained by more than 1 person, over timespans of months or longer? The cost/benefit calculus just doesn’t leave any room for debate.

                          1. 2

                            But static typing definitely improves many/most dimensions of project maintainability, compared to dynamic typing. This isn’t really a controversial claim!

                            On the contrary, it’s a very controversial claim.

                            Proponents of static typing like to just assert things like this without proof. But proof you must have, and thus far nobody has managed it – every attempt at a rigorous study to show the “obvious” benefits of static typing has failed. Typically, the ones that find the effect they wanted have methodological issues which invalidate their results, and the ones that have better methodology fail to find a significant effect.

                            The cost/benefit calculus just doesn’t leave any room for debate.

                            Again: prove it. WIth more than anecdata, because we both have anecdotes and that won’t settle anything.

                        2. 2

                          Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                          I see most typing errors as self-inflicted wounds at this point. Don’t have time or patience for things that can be prevented by the compiler happening at runtime.

                          Dynlangs + webdev together is my kryptonite. If I had to do that all day I’d probably start looking for a new career. Just can’t deal with it.

                      2. 1

                        I don’t know of any robust tooling for refactoring large JavaScript projects

                        Following up on this specifically: I’m an Emacs guy, not really an IDE person, so I don’t know the landscape that well. But everybody I know who goes the IDE route in Python uses PyCharm, so I looked up the rest of the JetBrains product line, and sure enough they have an IDE for JavaScript/TypeScript which claims to support refactoring.

                        I assume it’s not the only such product out there.

                      3. 3

                        Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.

                        Yes, projects can fail for lots of reasons; no one is claiming that static typing will make a shitty idea commercially successful, for example :) But I do think static types help a lot within their narrow scope–keeping code maintainable, reducing bugs, preserving development velocity, etc. Of course, there’s no good empirical research on this, so we’re just going off of our collective experiences. 🤷‍♂️

                        1. 1

                          Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.

                          I think it pretty much does, actually. Static typing moves an enormous class of invariants from opt-in runtime checks to mandatory compile-time checks. Statically typed languages in effect define and enforce a set of assertions that can be approximated by dynamically typed languages but never totally and equivalently guaranteed. There is a cost associated with this benefit, for sure, but that cost is basically line noise the moment your project spans more than a single developer, or extends beyond a non-trivial period of time.

                          1. 1

                            I think it pretty much does, actually.

                            As I said to your other comment along these lines: prove it. The literature is infamously full of people absolutely failing to find effects from static typing that would justify the kinds of claims you’re making.

                        2. 13

                          I always laugh when I see ruby code where the start of the method is a bunch of “raise unless foo.is_a? String”. The poor mans type checking all over the place really highlights how unsuitable these dynamic languages are for real world use.

                          1. 7

                            To be fair, any use of is_a? in ruby is a code smell

                            1. 12

                              Sure, it’s also a pattern I have seen in every Ruby codebase I have ever worked with because the desire to know what types you are actually working with is somewhat important for code that works correctly.

                              1. 5

                                Yeah, the need for ruby devs is much larger than the supply of good ones or even ones good enough to train the others. I’ve seen whole large ruby codebases obviously written by Java and C++ devs who never got ruby mentoring. I expect this is an industry wide problem in many stacks

                            2. 5

                              You seem to just be trolling, but I’ll play along, I guess.

                              I’ve seen a bit of Ruby, and a lot of Python and JavaScript, and I’ve never seen this except for code written by people who were coming from statically-typed languages and thought that was how everyone does dynamic typing. They usually get straightened out pretty quickly.

                              Can you point to some examples of popular Ruby codebases which are written this way? Or any verifiable evidence for your claim that dynamic languages are “unsuitable… for real world use”?

                              1. 5

                                I’m not trolling at all. I’ve been a Rails dev for the last 7 years and seen the same thing at every company. I don’t work on any open source code so I can’t point you at anything.

                                I quite like Rails but I’m of the opinion that the lack of static type checking is a serious deficiency. Updating Rails itself is an absolute nightmare task where even the official upgrade guide admits the only way to proceed is to have unit tests on every single part of the codebase because there is no way you can properly verify you have seen everything that needs to change. I’ve spent a large chunk of time spanning this whole year working towards updating from Rails 5.1 to 5.2. No one else dared attempt it before I joined because it’s so extremely risky.

                                I love a lot of things about Rails and the everything included design but I don’t see a single benefit to lacking types. Personally I see TypeScript as taking over this space once the frameworks become a little more mature.

                                1. 3

                                  You made a very specific assertion about how people write Ruby (lots of manual type-checking assertions). You should be able to back up that assertion with pointers to the public repositories of popular projects written in that style.

                                  1. 7

                                    I remembered hearing from my then-partner that Rails itself uses a lot of is_a?, and that seems true.

                                     if status.is_a?(Hash)
                                            raise ArgumentError, etc...
                                    
                                    1. 3

                                      This is pretty misleading – a quick glance at some of the examples seems like many of them aren’t really checking argument types, and when they are, they’re often cases where a method accepts any of multiple types, and there’s branching logic to handle the different options.

                                      Which is something you’d also see in a statically-typed language with sum types.

                                      The proposition that this is a common idiom used solely as a replacement for static checking is thus stil unproved.

                                      1. 1

                                        Well yeah, and then there’s those that raise errors, or return some failure-signaling value.

                                        I don’t know what projects to look at since I don’t use Ruby, but I found some more in ruby/ruby.

                                    2. 6

                                      ill concur with GP: this is a fairly common pattern to see in ruby codebases.

                                      however, to be fair, it’sa pattern most often introduced after attending a talk by a static typing weenie…

                                2. 4

                                  Do you also laugh when you see “assert(x > 0);” in typed languages?

                                  1. 6

                                    I would, but it would be a sad laugh because I’m using a type system that can’t express a non-zero integer.

                                    1. 2

                                      I would love to see broader adaptation of refinement types that let you statically guarantee properties like integer values being bound between specific values.

                                  2. 3

                                    I’m in the Type everything if it’s even kinda big camp now. There are too many things I need to think about during the day to remember the state and usage of every variable of every program I’ve ever written, used or inspected. Typings are rails for my logic. Typings are documentation. Types help my IDE help me. I will take every single shortcut I can when the timespan I or anyone else could be interacting with the code is longer than 10 minutes.

                                    Retracing steps is just so tedious and frustrating when you had it all in your head before. It just sucks. I just wanna build stuff, not fill my head with crap my computer can do.

                                    /rant

                                    1. 2

                                      I’m in the “I don’t make type errors, and if I do, I can shake them out with a few tests” camp for as long as I can comprehend what’s going on in the codebase.

                                      This is generally true for me, but writing tests or debugging stack traces makes for a slow iteration loop. A type error from a compiler usually contains better, more direct information so resolving these type errors is a lot faster. To the extent that I (a 15 year Pythonista) eventually began to prototype in Go.

                                      That said, the biggest advantage for me for a static type checker is that it penalizes a lot of the crappy dynamic code (even the stuff that is technically correct but impossible to maintain/extend over time). A static type system serves as “rails” for less scrupulous team members. Of course, a lot of these less-scrupulous developers perceive this friction as a problem with static type systems rather than a problem with the way they hacked together their code, but I think Mypy and TypeScript have persuaded many of these developers over time to the extent that static types are much less controversial in most dynamic language communities.

                                      Another big advantage is that your type documentation is always correct and precise (whereas docs in a dynamically typed language often go stale or simply describe something as “a file-like object” [does that mean it just has a read() method, or does it also need write(), close(), seek(), truncate(), etc?]). Further, because the type docs are precise, you can have thinks like https://pkg.go.dev complete with links to related types, even if those types are declared in another package, and you get all of this for free.

                                    1. 4

                                      I’m not sure how well ActivityPub will scale. The Mastodon server I’m on has a DB of over 200GB with 25K users - and it’s only been up for a few weeks. It seems like fairly soon it will hit the limits of what can be handled on a couple of home servers. I don’t know how much of this is Mastodon specific though.

                                      1. 11

                                        Let me assure you that the ActivityPub content is an insignificant part of that.

                                        Mastodon’s developers had the unfortunate (IMHO) idea to cache every piece of media it reaches a server on that server. The official explanation is that it alleviates the burden on smaller instances, so they don’t get hammered by thousands of requests for one piece of media that happens to go viral.

                                        However this makes it that every instance in the fediverse graph will have a full copy of every other piece of media of the other instances it federates with.

                                        This is not tenable, and in my opinion is working against the federation concept itself.

                                        I am working on software where serving dumb files and ActivityPub content that generated them is a low overhead affair, so even receiving hundreds of requests per second the performance won’t drop.

                                        1. 3

                                          Is the caching meant to alleviate network performance load or network/disk bandwidth load?

                                          1. 1

                                            I am not sure to be honest, I was presenting what I remembered from very old discussions (cca 2019,2020) - which might have been on github, or directly on Mastodon with Eugen and the other people in their dev team.

                                          2. 2

                                            Something combining CDN + media hosting + dedupe would be the obvious fix for this. There are github issues proposing IPFS, which probably hits the media hosting + dedupe part.

                                            1. 1

                                              I’m also considering using IPFS for media storage in the application I’m writing, but I’m not there yet. I’m prepared to let hyperlinking do its thing for a while. :)

                                          3. 2

                                            You should not expect a home server to handle 25K users. But the great thing about how the network is structured, is that it doesn’t have to be that way. I find that the smaller instances, of around 100 people, are a lot better, both in moderation quality, and in community.

                                            I am honestly disturbed how much people flock to the big servers, because this is coming back to centralize the network. I’ve already seen old-timers starting to block the biggest instances, because the moderation quality in them very noticeably dropped after Elon bought Twitter.

                                            1. 1

                                              I don’t think that’s a reasonable expectation, because that essentially boils down to ignoring everyone that isn’t tech savvy enough to roll their own instance, or lifting the burden onto each and every selfhosting capable IT guy.

                                              1. 2

                                                You don’t need to be tech savvy to run an instance, you just need a bit of cash to pay a hosting service.

                                                I realize not everyone has that either, but it’s a lot more people than have the time and energy to be their own host.

                                                1. 3

                                                  The same goes for everything where you are currently paying someone to make it work. For example your cloud, vegetables, messenger, email or your car.

                                                  And yes for many people this is on the same level. Truly federated, self hosted stuff is a pipe dream of us techies. It’s nice in theory, but a lot of burned out admins in their off time in reality if you really want to make it happen. I wouldn’t want to share random instances with folks I met one day, which I liked at that point, the same way people add and remove others on facebook and twitter. And there is so much more: Do all these instances have enough bandwidth ? Backups ? Privacy & Security things ? Stuff that really matter if you actually throw normal users in this.

                                                2. 2

                                                  You only need one tech-savvy person per 100 people to host an instance. In my country, around 2.5% of working force are described as “IT specialists” - so to host enough servers for the entire country, about 80% of them would need to host their own instance. A lot, but that’s still possible.

                                                  But you don’t need to be that tech-savvy to admin an instance - there’s plenty of different services offering hosting services. Last I heard, Masto.host hosts around 10% of all Mastodon instances by count, but other services, offering Mastodon or other software hosting exist. Using these services requires little technical knowledge.

                                                  1. 1

                                                    That is still more people than what traditional centralized social networks are offering.

                                                    Yes by all means make it easy to run, invest time into having a straightforward install and configuration, but asking a dev to make server software that your grandma’ can run is a little unreasonable to ask in my opinion.

                                              1. 11

                                                I think everyone’s experience is going to be different, this is all highly subjective depending on what communities you want to interact with. That being said, I’m very pleasantly surprised at how smoothly the transition to Mastodon has gone for my professional/programming feed.

                                                1. 14

                                                  I’m shocked that it worked (~70% of people I follow have moved to Mastodon, and it’s more active than Twitter now). It’s like the year of Linux desktop happening.

                                                  1. 9

                                                    It turns out all we need for the year of the Linux desktop to happen is for Elon Musk to buy a major competitor, say, Microsoft, or Apple. Doesn’t seem likely, but neither does a lot of stuff that is definitely happening in our timeline.

                                                    Maybe there is someone else (incredibly rich yet wildly incompetent and detached from reality) who could do Linux a similar service? It doesn’t literally have to be Elon Musk, it’s clear that wealth does not correlate with any ability or positive qualities whatsoever, there’s got to be hundreds of other (less famous?) plutocrats who also have the potential to destroy anything they touch. Thiel? DHH? Maybe a young con artist in the tradition of Theranos or FTX could convince investors to fund their joyride in the crumbling shell of a previously productive corporation.

                                                    1. 7

                                                      Doesn’t seem likely,

                                                      0% chance. Elon Musk bought Twitter for $44 billion. Apple and Microsoft are worth way more than that.

                                                      Apple market cap: $2.29 trillion (52 Twitters)

                                                      Microsoft market cap: $1.80 trillion (41 Twitters)

                                                      Even assuming either company would consider privatizing (they wouldn’t), even the top 10 richest people pooling their net worth couldn’t buy either of them.

                                                    2. 5

                                                      I’m on the one hand happy that it worked, but the move for me has been… spotty. Essentially my whole tech timeline moved, so that’s fine.

                                                      But I actively used Twitter to get into community I am very curious about. I have family in Botswana and so I used Twitter to stay up to speed on what happens in the region. All that migrated to about 0% and that’s a huge loss for me.

                                                    3. 3

                                                      I need to check it out again. When I’ve tried it in the past, it always felt like a bunch of people and bots shouting into the void–lots of content, but very little dialogue.

                                                      1. 7

                                                        lots of content, but very little dialogue

                                                        Twitter isn’t really a medium for dialogue anyway…

                                                        1. 1

                                                          Well, it’s not a platform that optimizes for constructive dialogue, so maybe a less ambiguous term would be “engagement”: there is a lot of engagement on Twitter, but very little on Mastodon (IME).

                                                        2. 4

                                                          Anecdotally, the Twitter Migration has caused a qualitative change, for better or worse. There is tons of dialogue on my instance.

                                                          1. 1

                                                            My biggest challenge right now is that my instance got too popular I guess and now is constantly down. :(

                                                            1. 1

                                                              What’s the bottleneck? Database? Web server? Something else?

                                                              1. 1

                                                                Oh sorry I meant the instance I’m on. That is, I don’t know what the problem is.

                                                          2. 1

                                                            If you’re active on Twitter I can recommend using one of the services that checks if your followers are also on a Fediverse instance, and allow you to easily import them.

                                                        1. 3

                                                          Its interesting to see the tension out in the community. Some of us, like this article, are arguing for more focus, more modularity, more maintainability, more re-use. But there is also a camp that misses the “good old days” where every C framework contains the kitchen sink and you just need one so file as a dependency.

                                                          1. 3

                                                            the “good old days” where every C framework contains the kitchen sink and you just need one so file as a dependency

                                                            A couple of years ago I worked on a project where including the whole of dlib was easier than rewriting/extracting a machine learning algorithm contained in it. As a bonus we got a whole network stack, an GUI library, an XML parser, and a bunch of compression algorithms.

                                                            The project ran fully as a web backend.

                                                            1. 2

                                                              meanwhile I’m sitting here with my 500+ packages for a rust app that is basically a CRUD, including markdown rendering, an ORM builder and some templating

                                                              but I can tell you half of these are duplicates regarding different features or versions of the same crate

                                                              guess it’s many “do one thing”s and most of them “do it right”

                                                              1. 4

                                                                Counting number of packages is mostly meaningless, because they vary in size and complexity. Big numbers imply bloat and excess, but whether that’s true is not possible to tell based on the number alone.

                                                                reqwest will pull in 150 packages, but if you look at them they’re all for reasonable things a modern full-featured HTTP client needs: HTTP protocol implementations, handling of sockets, URL parsing, TLS, gzip, brotli, JSON, base64, punycode, mime parsing.

                                                                OTOH curl is one dependency. It seems super frugal to use one dependency instead of 150. But did you know curl supports email? and gopher, ssh, samba, ldap, and more. It still has hundreds of “packages” worth of code, from many different authors, it’s just all bundled in the way you don’t see.

                                                                1. 4

                                                                  I agree with your point, but not with your example. curl is objectively huge but it’s also a poster child for projects that are well-maintained, especially regarding backwards compatibility. I think I don’t even remember a single time where a bumped curl dependency broke anything, so (from me) they kinda get a free pass.

                                                                  1. 1

                                                                    but that’s just another illustration of how problems with dependencies can’t be reduced to a single number, and avoidance of dependencies may not make sense.

                                                                    If you tried to replace curl with a “less bloated” library, you could have more issues with it. Or if you went all the way to claim you have “no dependencies!”, you’d have to reinvent a ton of code, likely with a much worse implementation, and a ton of headaches maintaining it all by yourself.

                                                                  2. 1

                                                                    OTOH curl is one dependency. It seems super frugal to use one dependency instead of 150. But did you know curl supports email? and gopher, ssh, samba, ldap, and more. It still has hundreds of “packages” worth of code, from many different authors, it’s just all bundled in the way you don’t see.

                                                                    Numbers obviously don’t paint the whole picture, and definitely not about bloat, but that doesn’t mean none of the small parts of the picture that they do paint aren’t relevant. In curl’s case, if you do work on a project that really does need email, gopher, ssh, samba and LDAP, having one dependency that supports all of them vs. one for each does mean that:

                                                                    • They expose a similar interface where possible, or at least generally use the same idioms, and share a lot of backend code
                                                                    • They generally have the same approach to feature lifetime (and, importantly for long-lived projects, deprecation)
                                                                    • There’s no effort in integrating them (they’re already integrated!)
                                                                    • You only need to monitor a single project’s mailing lists, you only deal with one release stream
                                                                    • If you commit fixes upstream, you have a single set of policies to deal with

                                                                    I don’t miss working with things like Boost, but depending on what’s in it, staying on top of a 100+-package dependency chain can be really exhausting in my experience.

                                                              1. 7

                                                                I noticed a weird trend where the check for if something is undefined happens after the undefined behavior. Intuitively it makes sense to me that this is insufficient so I’m wondering why this failure is so common? For example here the overflow check happens after the overflow already happened.

                                                                1. 4

                                                                  I don’t usually do that, but in my case there were 2 reasons:

                                                                  • the initial intention when writing it wasn’t protecting against overflow/UB but simply protecting against the computation going outbound for the dereference
                                                                  • the assignment needed to be done early because I was targeting a codebase with some ancient C convention on variable declaration required to be before any code; and since I had the form where I wanted to bail out early instead of opening a scope, I had to declare it early:
                                                                  int x = ...
                                                                  if (...)
                                                                      return -1
                                                                  // x can not be declared here
                                                                  return f(x)
                                                                  
                                                                  1. 9

                                                                    Not trying to be critical, but it shouldn’t be news that you can’t check for a UB condition after it’s happened. I’ve seen so many cases where people run into similar problems, or problems related to the type-punning rules. Usually the thought process is something along the lines of:

                                                                    ok, so I know this is supposed to have undefined behavior. But the variable will still have some value, so I can check if the value is in the correct range and so avoid issues that way…

                                                                    No, you can’t. This is what undefined behaviour means. All bets are off; if it’s hit, it is a bug in the code, full-stop, and no checking afterwards can fix it because the behaviour from that point (*note1) on is totally undefined. Maybe it seems to work in some cases. It doesn’t matter; use a different compiler, or a later version of the same compiler, and all of a sudden it could stop “working”.

                                                                    Don’t think of the C compiler as some sort of glorified high-level assembler. It’s way more sophisticated than that. There are rules you have to follow, if you are using any of the modern compilers. You don’t have to like it (and there are even switches available that will give behaviour that you want, but that’s not standard C any more) but it is the case. You must not ever invoke undefined behaviour.

                                                                    Note 1: Actually, I believe the whole program behaviour is undefined if the program exhibits undefined behaviour at any point. So technically, even things that were supposed to happen before the UB might not happen or might happen differently.

                                                                    1. 5

                                                                      You are technically correct, but I’m sure you understand that the consequences of such a policy means that pushed to the extreme we could have the situation where a 100k LoC codebase has a single little bug deep down somewhere, then crashing or executing random code straight at startup is an acceptable behavior.

                                                                      The cost of a single mistake is very high, that’s the main point I’m trying to make.

                                                                      1. 9

                                                                        What’s the alternative? If the compiler can’t optimise around code that hasn’t been executed yet having UB, then the opportunities for useful optimisation become near non-existent.

                                                                        The compiler is not doing anything unreasonable here: every step, in isolation, is desirable and valid. If the end result feels unreasonable, then that’s either (a) a problem with the language spec being insufficiently relaxed about what it considers to be UB or (b) a problem with insufficient tooling (or, in the case of a language like Rust, built-in compiler checks) to catch issues like this.

                                                                        To point the finger at the compiler is a very hard sell indeed because there’s no specific thing to point at that it’s doing wrong.

                                                                        1. 9

                                                                          It might be reasonable not to do the optimization. The alternative in rust is to actually define the behavior of wrapping, which would be equivalent to using -fwrapv in C. Sure we loose some optim, but is it worth it? I’m starting to believe so.

                                                                          1. 10

                                                                            Yes, I agree: but that’s a problem with the language spec, not the compiler. The language spec should just say ‘overflow has wrapping semantics’. You’ll lose some opportunities for optimisation and compatibility with a lot of older of obscure platforms (some platforms have arithmetic instructions that don’t wrap on overflow, and this is one of the big reasons that the C spec leaves overflow undefined!), but this is enough of a footgun that I think it’s a worthwhile tradeoff in the year of our lord 2022.

                                                                            But again, this isn’t GCC’s fault: this is the fault of the language spec and the compromises that went into its creation. Don’t like it? Time to get a new language (this isn’t me trying to be gatekeepy: horrid footgun shit like this is a big reason I moved to Rust and never looked back).

                                                                            1. 6

                                                                              Not saying it’s GCC fault, but just because a spec did a mistake doesn’t mean GCC should be braindead about it: it holds a responsibility for all the software in C out there. Nothing forces GCC to do dangerous optimizations; it can still follow the specs by not honoring this part. GCC serves the user, not the specs; the question becomes: do users want this kind of optimization and assume its consequences by default?

                                                                              1. 3

                                                                                Where’s the mistake? Integer overflow being undefined is a feature, not a bug. There are platforms where the behaviour of overflow is implementation defined, entirely unpredictable, or just straight up UB at a hardware level, leaving the machine in a totally invalid state. C is designed to target bizarre and unusual architectures like these, and so having integer overflow be undefined is a huge boon to the language’s portability without sacrificing (and even improving, in many cases) performance.

                                                                                If you’re just going to do language spec revisionism and claim that ‘the spec is wrong’ or something, then I think it’s clear that C’s not the language for you. Heck, it’s definitely not the language for me: I aggressively avoid touching the thing nowadays.

                                                                                1. 3

                                                                                  I am sure there is, so please name one.

                                                                                  1. 3

                                                                                    Many GPUs have saturating semantics on overflow. Other architectures emulate small integers with large integers, meaning that overflow results in unobservable ‘extra bits’. Changing the standard to make integer overflow always wrap would make writing C for these architectures extremely difficult without significant performance ramifications.

                                                                                    If reducing portability is fine with you, then so be it: but that’s not what C is for: it’s supposed to be the lowest common denominator of a vast array of architectures, and it does this quite effectively in no small part because it leaves things like integer overflow undefined.

                                                                                  2. 3

                                                                                    There are platforms where the behaviour of overflow is implementation defined, entirely unpredictable, or just straight up UB at a hardware level, leaving the machine in a totally invalid state.

                                                                                    Can you name one such platform? That is still used after Y2K?

                                                                                    Also note that the spirit of UB in 1989 was almost certainly a compatibility thing. I doubt the standard committee anticipated anything other than -fwrapv on regular 2’s complement processors. And it’s only later that compiler writers realised that they could interpret “UB” in a way that in this particular case was most probably against the spirit of it.

                                                                                2. 2

                                                                                  Yes, I agree: but that’s a problem with the language spec, not the compiler.

                                                                                  Compiler writers are on the standard committee…

                                                                                  1. 1

                                                                                    I dont think defining the behaviour of overflow is desirable: programmers want overflow to happen in very rare cases and defining its behaviour now means tools cannot distinguish between overflow the programmer wanted/expected and accidental overflow (the vast majority of cases in my experience).

                                                                                    We currently can write sanitizers around overflow because its undefined, if we had defined it as wrapping the sanitizers could only say “well its wrapping, but I guess you wanted that, right ?”

                                                                                    AFAIU rust traps on overflow in debug, and defines it as wrapping in release, I believe this is mostly because they decided undefined behaviour in safe code was unacceptable, so they went with defined but very likely wrong in release.

                                                                                  2. 4

                                                                                    You lose far fewer optimisations in a language that is not C. Unfortunately, in C, it is a very common idiom to use int as the type for a loop induction variable. Having to reason about wrapping breaks a lot of the analyses that feed vectorisation. In C++ or Rust, you typically use iterations, rather than raw indexes, and these iterations will use an unsigned type by default. Operating over the domain of positive integers with wrap to zero is much simpler than operating over the domain of signed integers with overflow wrapping to a large negative number and so the C++ and Rust versions of these loops are easier to vectorise. In C, using something like size_t as the type of the induction variable will often generate better code.

                                                                                    1. 2

                                                                                      Then… how about renouncing these optimisations, and tell everyone to update their code to use size_t so it is fast again? Because I sure resent compiler writers for having introduced critical vulnerabilities, and tell everyone to fix their programs so they are safe again…

                                                                                      I mean, sometimes the hoops I have to jump through… libsodium and Monocypher for instance can’t use arithmetic left shifts on signed integers at all. Instead of x << 26 we need to write x * (1<<26), and hope the compiler will be smart enough to generate a simple shift (thankfully it is). Reason being, left shifting negative integers is straight up undefined. No ifs, no buts, it’s undefined even when the result would stay within range.

                                                                                      1. 3

                                                                                        Then… how about renouncing these optimisations

                                                                                        That’s what PCC does. It renounces all of these optimisations and, as a result, generates much slower code. OpenBSD tried to use it for a while, but even they couldn’t put up with the poor performance (and OpenBSD generally favours security over performance). The market has shown time and time again that a C compiler that generates fast code will always be chosen over one that generates more predictable code for undefined behaviour.

                                                                                        It’s not like there aren’t alternative compilers that do what you claim to want, it’s just that no one (including you) actually wants to pay the performance cost of using them.

                                                                                        1. 3

                                                                                          The market has shown time and time again that a C compiler that generates fast code will always be chosen over one that generates more predictable code for undefined behaviour.

                                                                                          Gosh, I think our mutual employer provides a strong counter-example. The incentives of a standalone compiler vendor are very different to a vendor that uses the compiler to compile billions of lines of its own production code. Our compiler adds new security features at the expense of performance continually, and internal code is required to use them. IMHO these end up at the opposite absurd end of the spectrum, like default-initializing stack variables to ensure the undefined behavior on access becomes implementation defined, stack overflow buffer checks, etc. In addition to performance vs. security, there’s also a stronger emphasis on compatibility vs. performance; updating the compiler in a way that would defeat large numbers of existing security checks would come under a lot of scrutiny.

                                                                                          1. 2

                                                                                            I thought about MSVC’s interpretation of volatile as a counter example here (it treats it as the equivalent of sequentially consistent atomic, because that’s what a lot of internal legacy code assumed). But then I thought of all of the third-party project switching to using clang on Windows, including things like Chrome and, by extension, all Electron apps and realised that it wasn’t such a counter example after all. For a long time, MSVC was the only compiler that could fully parse the Windows headers, which gave it a big advantage in the market, now that clang can do the same, that’s eroding (I believe clang can now parse all of the Windows source code, but it can’t correctly codgen some large chunks and doesn’t generate code that is the shape expected by some auditing tools).

                                                                                            Alias analysis is another place where MSVC avoids taking advantage of undefined behaviour. Apple pushed for making -fstrict-aliasing the default and fixed (or encouraged others to fix) a huge amount of open source and third-party code, giving around 5% better performance across the entire system. MSVC does not take advantage of type-based alias analysis because doing so would break a lot of existing code that relies on UB. This is also pushing people who have code that does not depend on illegal type punning to use clang and get more performance.

                                                                                            Note that I am talking specifically about interpreting the standard with respect to UB to enable optimisations here. I see security flags such as /GUARD, stack canaries, InitAll, and so on as a different category, for three reasons:

                                                                                            • They are opt in, so you can ignore them for places where you know you’re sandboxing your program or where it isn’t operating on untrusted data.
                                                                                            • They make certain behaviour well defined, which makes it easier to reason about your source code. Not taking advantage of UB does not have this property: your program still depends on UB and may still behave in unexpected ways and your performance is now harder to reason about because it will vary hugely depending on whether, post inlining, you have enough hints in your source for the compiler to prove that the condition will not trigger.
                                                                                            • They, in general, don’t impede other optimisations. For example, InitAll combines with stead store elimination and typically can be elided by this optimisation (and Shayne did some great work to improve this). /GUARD is applied very late in the pipeline (I worked on the LLVM implementation of this so that we could have CFG for Rust and Objective-C), and so inlining and devirtualisation can significantly reduce the number of places where you need the check (MSVC has some very impressive profile-guided devirtualisation support, which helps a lot here). In contrast, things like not assuming that integer addition results in a larger number have a big knock-on effect on other optimisations.
                                                                                          2. 1

                                                                                            Well, there is renouncing a class of optimisations, and defining a class of behaviours. I don’t think those are the same. Which one PCC was trying? Did it define integer overflow and pointer aliasing etc. or did it disable dangerous looking optimisations altogether?

                                                                                            it’s just that no one (including you) actually wants to pay the performance cost of using them.

                                                                                            I put myself in a situation where I can actually cop out of that one: I tend to write libraries, not applications, and I ship source code. This means I have no control over the compilation flags, and I’m forced to assume the worst case and stick to strictly conforming code. Otherwise I would try some of them (most notably -fwrapv) and measure the impact on performance. I believe I would accept any overhead below 5%. But you’re right, there is a threshold beyond which I’d just try to be more careful. I don’t know for sure which threshold this is though.

                                                                                            1. 1

                                                                                              I tend to write libraries, not applications, and I ship source code. This means I have no control over the compilation flags

                                                                                              How’s that? Libraries would still come shipped with a build system to produce (shared) objects, right?

                                                                                              1. 1

                                                                                                Libraries would still come shipped with a build system to produce (shared) objects, right?

                                                                                                Not when this library is literally one C source file with its header, with zero dependency, and used on obscure embedded targets that don’t even have a standard library and I don’t know of anyway.

                                                                                                I do ship with a Makefile, but many people don’t even use it. And even if they did, they control $CFLAGS.

                                                                                                1. 1

                                                                                                  Ouch, that’s not an enviable situation to be in :S

                                                                                                  Too bad you can’t enforce some of those semantics using #pragma or something.

                                                                                                  1. 1

                                                                                                    Well, then again, I did it on purpose: sticking to standard C99 with zero dependency is how people ended up using it in those contexts. My work is used on a previously underserved niche, that’s a win.

                                                                                                    And in practice, I did one error of any consequence, and it was a logic bug, not anything to do with C’s UB. I did have a couple UB, but none ended up amounting to anything. (There again, it helps that my library does zero heap allocation.)

                                                                                  3. 6

                                                                                    Yes, that is exactly the by-design consequence of C UB. A single UB anywhere deep in your code could convert your computer into a giant whale or a potted plant.

                                                                                    1. 4

                                                                                      Yes. Writing code in C is a minefield, and I think people who write code in this language need to be aware of that.

                                                                                  4. 3

                                                                                    the assignment needed to be done early because I was targeting a codebase with some ancient C convention on variable declaration required to be before any code

                                                                                    If this is referring to C89 style, then you can declare a variable without assigning it:

                                                                                    int x;
                                                                                    if (...) { return -1; }
                                                                                    x = 123;
                                                                                    
                                                                                    1. 3

                                                                                      Yeah but I don’t like that for 2 reasons:

                                                                                      • 2 lines instead of 1
                                                                                      • I can’t do const int x = … anymore (and I like to use const everywhere because it helps the developer mental model about non-mutability expectations)
                                                                                  5. 4

                                                                                    Good observation. In C/C++ you are intended to check for valid preconditions before you perform an operation that relies on them. In Python and many others, there is a pervasive “look before you leap” idiom because there is no undefined behavior, either it behaves correctly or throws an exception, i.e. every operation is checked beforehand. Could be from an influx of folks into C/C++ from those languages.

                                                                                    For those who don’t understand, C/C++ does it this way because specifying “undefined behavior” allows you to assume that preconditions are valid without having to recheck them on every call, allowing the programmer to be more efficient with the CPU.

                                                                                    1. 3

                                                                                      In Python and many others, there is a pervasive “look before you leap” idiom because there is no undefined behavior, either it behaves correctly or throws an exception, i.e. every operation is checked beforehand.

                                                                                      I think “look before you leap” (LBYL) is the opposite of what you’re trying to describe. I’ve usually heard that described as “easier to ask forgiveness than permission” (EAFP).

                                                                                      1. 1

                                                                                        My mistake, I meant “leap before you look”

                                                                                    2. 1

                                                                                      Note that the order of operations doesn’t matter for UB. UB is not an event that happens. Instead, “UB can’t happen” is an assumption that the compiler is free to make, and then move or delete code under that assumption. Mere existence of any UB anywhere in your program, even in dead code that is never executed, is a license to kill for a C compiler.

                                                                                      1. 1

                                                                                        even in dead code that is never executed, is a license to kill for a C compiler.

                                                                                        No, unless you mean that it’s a license to remove the dead code (which the compiler can do anyway).

                                                                                        If code that would have undefined behaviour when executed is never executed, then it does not trigger the undefined behaviour (by definition).

                                                                                        1. 1

                                                                                          Whole-function analysis can have an effect that seems like UB going back in time. For example, the compiler may analyze range of possible values of a variable by checking its every use and spotting 2 / x somewhere. Division by 0 is UB, so it can assume x != 0 and change or delete code earlier in the function based on this assumption, even if the code doesn’t have a chance to reach the 2 / x expression.

                                                                                          1. 2

                                                                                            For example, the compiler may analyze range of possible values of a variable by checking its every use and spotting 1 / x somewhere, and then assume x != 0 and change or delete code based on that earlier in the function, even before execution has a chance to reach the 1 / x.

                                                                                            Yep, but if that 1 / x is in dead code it can’t affect assumptions that the compiler will make for live code. And if the 1 / x is in a particular execution path then the compiler can’t use it to make assumptions about a different path.

                                                                                            As an example, for:

                                                                                            if (x == 0) {
                                                                                                printf("x is zero!\n");    
                                                                                            }
                                                                                            
                                                                                            if (x == 1) {
                                                                                                printf("1/x = %d\n", 1 / x);
                                                                                            }
                                                                                            

                                                                                            … the compiler will not remove the x == 0 check based on division that occurs in the x == 1 branch. Similarly, if such a division appears in dead code, it can’t possibly affect a live execution path.

                                                                                            So:

                                                                                            even in dead code that is never executed, is a license to kill for a C compiler.

                                                                                            No.

                                                                                            (Edit, based on your edits): In addition:

                                                                                            Division by 0 is UB, so it can assume x != 0 and change or delete code earlier in the function based on this assumption,

                                                                                            Yes, if it is guaranteed that from the earlier code the 2 / x division must be subsequently reached, otherwise no.

                                                                                            even if the code doesn’t have a chance to reach the 2 / x expression.

                                                                                            No. As per above example, the compiler cannot assume that because something is true on some particular execution path it is true on all paths.

                                                                                            If what you were claiming was true, it would be impossible/useless to perform null checks in code. Consider:

                                                                                            if (p != NULL) {
                                                                                                *p = 0;
                                                                                            }
                                                                                            

                                                                                            If the compiler can assume that p is not NULL based on the fact that a store to *p exists, it can remove the NULL check, converting the above to just:

                                                                                            *p = 0;
                                                                                            

                                                                                            This is clearly different and will (for example) crash if p happens to be NULL. But a compiler can’t and won’t make that change: https://godbolt.org/z/hzbhqdW1h

                                                                                            On the other hand if there is a store that appears unconditionally on the same execution path it can and will remove the check, eg.

                                                                                            *p = 0;
                                                                                            if (p != NULL) {
                                                                                                printf("p is not null!");
                                                                                            }
                                                                                            

                                                                                            … for which both gcc and clang will remove the check (making the call to printf unconditional): https://godbolt.org/z/zr9hc7315

                                                                                            As it happens, neither compiler will remove the check in the case where the store (*p = 0) is moved after the if block, but it would be valid for them to do so.

                                                                                      2. 1

                                                                                        I think this is the core of the issue and why people are getting so fired up.

                                                                                        If you assume that integer operations are sent to the CPU in tact, and the CPU was made in the last 30 years, then checking for overflow after the fact is a single compare.

                                                                                        If you have to check for the potential for overflow beforehand, the comparison is much more involved. I was curious what it actually looks like and stumbled onto this which implements it in four compares (and three boolean evaluations.)

                                                                                        At some level, this whole conversation becomes a disagreement about the nature of bounds checking. If you assume bounds checking does not exist (or can be compiled away!) then you can exploit UB to optimize signed arithmetic to improve performance. If you assume bounds checking needs to exist, that UB exploit is a huge negative because it forces much more pessimism to put the bounds check back, making performance worse.

                                                                                        Then we end up with compiler builtins to perform signed arithmetic deterministically. This is odd because the UB optimization assumes that if the language spec doesn’t require something it’s not needed in an implementation, but the existence of the builtin suggests otherwise. The UB optimization assumes that there’s no value in having a predictable implementation defined behavior, but the builtin is a predictable implementation defined behavior.

                                                                                        1. 1

                                                                                          It’s the observation that most of the time overflow is a bug. If you wanted overflow semantics, you should have asked for it specifically. This is how e.g. Zig works.

                                                                                          1. 1

                                                                                            Avoiding the overflow requires a bounds check. I interpreted your earlier question being about why these bounds checks often create an overflow in order to perform the check (which is not a bug, it’s integral to the check.) There’s no standards compliant way to request overflow semantics specifically, so that option doesn’t really exist, and doing the check without an overflow is gross.

                                                                                            If the standard had provided a mechanism to request overflow semantics via different syntax, we probably wouldn’t have such intense discussion.

                                                                                            1. 1

                                                                                              I agree, not having a checked add or a two’s complement add is definitely a hole in the standard and should be fixed.

                                                                                      1. 3

                                                                                        Something along the same lines that I’ve been doing (in OO languages): Reducing usage of instance variables.

                                                                                        Even when an instance variable would be very much accessible in a given method I’m writing, I write the method so that it has to take in another argument instead, which serves the instance variable’s purpose. Similar reasoning as with global variables, but on a smaller scale: It reduces the surface area of what (in the class’s code) could possibly impact what the method in question does, thereby making it easier to read and reason about.

                                                                                        Separate I/O from processing

                                                                                        I’d have appreciated an example of this, as I don’t quite get the point the author is trying to make. “Here’s an example where it’s not separate. See how these problems can occur? Now here’s what it looks like when we separate them.”

                                                                                        1. 4

                                                                                          Separating I/O from processing is good instinctual starting point.

                                                                                          Not all I/O code is a file read, sometimes it’s and RPC, or a Database call, or listening on a socket.

                                                                                          Having your core logic not worrying about the details of how the data gets to it allows you to reuse it across more domains, and allows your I/O code to focus on I/O, rather than having to think of both at the same time.

                                                                                          That being said, that are times when you have to make sure your separation of concerns doesn’t lead to bad I/O patterns, like transaction-per-row database interactions.

                                                                                          And you have things like ETL(Extract, Transform, Load) flows, where the I/O is the processing, for the most part.

                                                                                          1. 3

                                                                                            If the method works without any instance variables, why is it even in the class at all?

                                                                                            1. 1

                                                                                              That’s a fair question. A lot of the time, it’s private functions that do some intermediary work that doesn’t need to be exposed as part of the class’s public interface.

                                                                                              1. 1

                                                                                                Usually I like to bundle such things into a module of related helpers, or else build a sub-object for that inner functionality.

                                                                                                Not that a method with no self references is like dogmatically evil, but it feels like a smell often.

                                                                                          1. 2

                                                                                            I agree with most of this strongly. My C programming got much less buggy and more maintainable when I started avoiding side effects like mutating global variables or reference parameters.

                                                                                            Separating I/O from processing is probably also a heuristic I tend to use that has made my programming better. If a prototype I make has I/O sprinkled in several places one of the first refactors I tend to do is consolidate or separate it out. I/O is where you interact with things outside your program, and having those interfaces separated more discretely makes them easier to debug, easier to test, and makes it easier to separate a “your code problem” from an interface or “their code problem”.

                                                                                            It was extremely frustrating though, in Haskell, to jump through hoops to do debug printing inside a function.

                                                                                            1. 7

                                                                                              GHC Haskell includes a purity-breaking “Debug.Trace” module to solve this problem if you’re ever in that space again

                                                                                            1. 14

                                                                                              Gleam is my favorite language that I haven’t actually tried yet (but I will as soon as I get home from my traveling).

                                                                                              I read the language tour on my phone one evening, and the next evening I was reading and understanding source code. That’s how approachable the language is!

                                                                                              use is interesting because it’s a fairly complex feature, but it opens up so many options (as shown in the link). At first I was skeptical, but I think I agree. It’s worth the added complexity.

                                                                                              To give two examples of where Gleam has made (imo) interesting design decisions to keep the language simple:

                                                                                              • There is no Option[T], only Result[T, E]. If you don’t want an error, just use Result[T, Nil]. Is it more typing? Sure, but now you only have one type instead of two.
                                                                                              • A type (think struct) can have multiple constructors, each with their own fields. This means they effectively double as enums. Have a look at the docs, they explain it better than I can.

                                                                                              Anyway, congrats Louis! Very inspiring work :-)

                                                                                              1. 4

                                                                                                A type (think struct) can have multiple constructors, each with their own fields. This means they effectively double as enums. Have a look at the docs, they explain it better than I can.

                                                                                                This sounds like it’s taken straight from Haskell. (Not a criticism, just background.)

                                                                                                1. 4

                                                                                                  One could do worse things then take inspiration from Haskell!

                                                                                                  I did not know this. :-)

                                                                                                  1. 4

                                                                                                    It’s not specifically inspired by Haskell, it was mostly inspired by Erlang. I didn’t realise that Haskell had the same feature but now I read the documentation it seems very similar. Cool stuff!

                                                                                                    1. 3

                                                                                                      I mean, Gleam does belong to the ML family of languages IMO, so we may as well say the feature is inspired by Standard ML! /s

                                                                                                      1. 1

                                                                                                        In what sense does it relate to ml?

                                                                                                        1. 6

                                                                                                          Some examples of “ML-like” features include…

                                                                                                          • Sum types as a primary data modeling tool, in some cases even displacing records for small-ish data structures

                                                                                                          • First-class functions and some use of recursion, often used to constrain and thereby elucidate control flow. This is sometimes in lieu of imperative-/procedural-style constructs such as…

                                                                                                            • If/else statements (not if/else expressions)
                                                                                                            • Switch/case statements (not if/else expressions)
                                                                                                            • Goto, which has basically been a strawman against this style of programming for the past decade or so
                                                                                                            • Defer, which though it allows code to be written out-of-order, still introduces a discrete procedural “step” to computation

                                                                                                            In languages like Javascript and Gleam, this extends to the use of a syntactic construct you may know as a callback function.

                                                                                                          • Type inference (usually Damas–Hindley–Milner type inference)

                                                                                                          • Other idioms that, like the above, help to obviate and/or discourage the use of global state in implementations as they scale

                                                                                                          There are plenty of ML-like languages for different runtimes, including a few that are used for systems programming.† Languages often described as “ML-like” include…

                                                                                                          • Scala, ‘an ML for the JVM
                                                                                                          • F#, ‘an ML for the CLR
                                                                                                          • Elm, which also takes some inspiration from Haskell while not going quite as far with the generics, apparently for sake of error message readability
                                                                                                          • Facebook’s Reason, which is sometimes even called ReasonML for clarity
                                                                                                          • Rust, one of the only systems programming languages to have this distinction. Check for the features above if you don’t believe me!

                                                                                                          Haskell is explicitly inspired by ML, but is often considered its own category due to the radical departures it makes in the directions of a) purely functional programming and b) requiring (in most cases) the use of monads to represent effects in the type system.


                                                                                                          My educated guess: This is largely because the core syntax and feature-set is relatively well understood at this point. As such syntactic sugar is rarely necessary in order to express intent directly in efficient code. This is unlike in “dynamic” languages such as Python, Ruby, and Elixir, which tend to make liberal use of metaprogramming in order to make the syntax more directly express intent. This can often make it unclear what is actually happening to make a given piece of code run.

                                                                                                          1. 3

                                                                                                            I find it interesting that nothing on that list is inherent to functional languages. All these sweet goodies might as well exist in an imperative language, but outside of Rust, they don’t.

                                                                                                            I’m still sad Go completely missed the boat on that one.

                                                                                                            1. 1

                                                                                                              Yup. Maybe someday we’ll have an approach in between those of Go and Rust; that’s some of what I’m looking to Gleam for, even if it’s not primarily a systems programming language.† In the meantime, we have the sometimes-insightful, sometimes-regressive minimalism of Go; and the sometimes-insightful, sometimes-overwhelming maximalism of Rust.


                                                                                                              † It is my broad understanding that the BEAM VM—on which I expect most* Gleam code will run—helps concurrent applications scale by denying granular control of threaded execution. This can be thought of as vaguely similar to the Go runtime’s decision to have a blessed implementation of cooperative multitasking, namely goroutines. In contrast, the Erlang ecosystem benefits from having a blessed provider and model (respectively) for all concurrent computation, thanks to the OTP supervision tree combined with the actor model of concurrent computation. It takes power away from the developer, but to the benefit of the system at large. Boy, is it ever exciting that we might actually have a statically-typed language built atop that excellent multitasking infrastructure!

                                                                                                              1. 1

                                                                                                                From my PoV Go and Rust are incomparable since Go is a GC’d language and Rust is a non-GC’d language. So on this basis Gleam, too, can never compete with Rust but is for sure a strong contender in the space Go plays in.

                                                                                                                1. 1

                                                                                                                  If all that matters is the engine that runs your code, the sure. If a project is performance-sensitive, then its options for reasonable programming language are constrained anyway IMO. When I compare these languages, I have their developer experience in mind. At least, relative to how much performance they leave on the table.

                                                                                                              2. 1

                                                                                                                It honestly depends what one even means by “functional language” lots of ML and ML-like things exist which are not particularly functional including possibly: SML, OCaml, Swift, Scala, Kotlin, Haxe

                                                                                                                1. 2

                                                                                                                  What’s not-functional about SML, OCaml, and Scala? Are you perhaps comparing them to the purely functional Haskell?

                                                                                                                  1. 1

                                                                                                                    What’s functional about them? Every language I listed is equally functional and not-functional depending how you use it. They’re all ML variants after all.

                                                                                                              3. 2

                                                                                                                Facebook’s Reason, which is sometimes even called ReasonML for clarity

                                                                                                                Credit where credit is due: Reason is a syntax extension for OCaml. It’s not a whole cloth creation by Facebook.

                                                                                                              4. 3

                                                                                                                I think it is an ML language in every way, although we moved away from the traditional ML syntax in the end.

                                                                                                        1. 13

                                                                                                          A server should not be treated as a community. The whole fediverse is out there. Treat it like you would email: ideally host on a domain you control, if you can’t then choose a host based on competence and stability and scaling etc. You can follow and interact with anyone on any server after that.

                                                                                                          1. 3

                                                                                                            Yeah, istm that blog + email = activity pub. It’s fine to use blog on BlogSpot or Medium if you don’t want the hassle of self-hosting, but if you’re serious about blogging, you should have your own domain name. Ditto email, if you’re serious you should own a host name, even if you outsource the hosting.

                                                                                                            1. 1

                                                                                                              No, a server really is a community. I hear what you’re saying, there’s no walls, except where server admins have created them. However, lots of people browse the local feed, a feed of all messages sent by people on that server. That means people will hear you even if they don’t follow you. One of the more frequent comments of newcomers is “wow, posting on mastodon isn’t like shouting into the void”, and this really is the reason. It’s a lot less lonely if you choose a good server.

                                                                                                              Hachyderm and fosstodon both have dense clusters of top-notch tech people. I’ve occupied both, and posting anything remotely interesting generates immediate chatter that leads to great conversations.

                                                                                                            1. 3

                                                                                                              Working on a budget. Comparing Firefly III and Actual (both running in the homelab) to try and decide which will be more likely to stick in the long term.

                                                                                                              Thinking about adding an instance of something that speaks ActivityPub to my fly.io apps, primarily looking at Pleroma or Akkoma. However, I’d really like to find one that was built to work with sqlite, so I can avoid running a second app for the database.

                                                                                                              1. 3

                                                                                                                You can run postgres and Pleroma both in the same app on fly. I do this

                                                                                                                1. 2

                                                                                                                  That makes sense; how do you accomplish this in your setup? For my CI/CD server, I used [processes] in fly.toml to run the web server and the Docker build agent inside a single app, I guess I could do the same thing by making a container image with postgres and Pleroma?

                                                                                                                  1. 3

                                                                                                                    Here’s my fly app if you like https://git.singpolyma.net/soapbox-fly/tree

                                                                                                                2. 3

                                                                                                                  FF3 doesn’t handle multiple currencies (exchange rate) and assets. This is the reason I stuck with ledger3 although I would like to give GNUCash a try.

                                                                                                                1. 6

                                                                                                                  Very handy and to the point! I appreciate you writing it.

                                                                                                                  My only small quibble is that HVEC is a proprietary codec. MDN’s video codec guide recommends using a WebM container with VP9 for video and Opus for audio, for everyday videos.

                                                                                                                  1. 1

                                                                                                                    Thanks for your feedback. I’m very interested on your quibble. Have already tried to upload a video with WebM container with VP9 for video and Opus for Audio ? Do you have any recommended FFmpeg command parameters to produce a file like that so that I can try locally and than update my article ?

                                                                                                                    1. 3

                                                                                                                      I have made quite a few video clips encoded with VP9, but AAC (edit: double checking my files, ffmpeg seems to have transcoded audio to Opus automatically) as audio and my experience is that a lot of devices have shoddy support for VP9. For the ones that support it, it is great, but for the ones where it doesn’t then you will need a fallback to h264. Depends on your use case here.

                                                                                                                      Regarding the conversion parameters I often convert h264 encoded videos to VP9 using these settings: ffmpeg -i <input_file> -c:v libvpx-vp9 -crf 18 -b:v 0 <output_file>

                                                                                                                      Works fine for me in my case, but I’ve never tried to upload them to mastodon, so… YMMV I guess?

                                                                                                                      1. 2

                                                                                                                        Or VP8 which will be a bit bigger but has broader performant decoding support.

                                                                                                                        1. 1

                                                                                                                          This is the reasonable option…

                                                                                                                        2. 1

                                                                                                                          Thanks for these details from your experience. Do you think you can try to upload a file generated that way on mastodon and see if it works ?

                                                                                                                          1. 1

                                                                                                                            I don’t have an account, but I can PM you a link to a file you can test with that is encoded in that way that you could test with if you want.

                                                                                                                        3. 2

                                                                                                                          I have not tried it, actually; I will today, if you’d like. In the process, I’ll figure out the FFmpeg parameters needed.

                                                                                                                          1. 1

                                                                                                                            Thank you very much, that would be helpful. I’m open to update my article based on the MDN’s recommendations.

                                                                                                                            1. 4

                                                                                                                              I’ve uploaded a one (1) minute clip to: https://emacs.ch/@carcosa/109365376767283805

                                                                                                                              The command-line I used was:

                                                                                                                              ffmpeg -i 'Sacrifice of Angels Battle.mp4' -vf scale=1920:-1 -c:v libvpx-vp9 -crf 24 -b:v 2000k -row-mt 1 -c:a libopus -b:a 48K 'Sacrifice of Angels Battle.webm'

                                                                                                                              1. 2

                                                                                                                                The audio is way out of sync, dunno if that’s your fault or the source material

                                                                                                                      1. 6

                                                                                                                        I really wish the only community options weren’t Discord and GitHub – not even an unofficial Matrix.org room or IRC channel mention (seems IRC used to be supported). I was looking into the language recently as something with types on BEAM, but I felt like there was no option to ask questions toward the community if I had them so I didn’t feel it was worthwhile to read any further.

                                                                                                                        1. 12

                                                                                                                          We did used to use IRC and we looked into Matrix but both lacked the anti-harassment and anti-abuse features that we need to run a community. Community is very important to Gleam so we need to ensure that we can deal with trolls and bigots should they arise.

                                                                                                                          As a nice unexpected bonus the active community exploded in size once we switched from IRC, it seemed it was a barrier to entry for many people.

                                                                                                                          1. 3

                                                                                                                            Is there a downside for listing unofficial options for people to join at their own risk if they don’t feel comfortable giving their private data to Discord?

                                                                                                                            1. 5

                                                                                                                              It’s tricky. So far the experience with unofficial projects (not unofficial community forums, afaik there are none) is that people still come to the Gleam core team when they want help with them or have a problem to report. It ends up creating additional work for us and we have quite limited time due not having corporate backing.

                                                                                                                              It would be good to list unofficial forums but we probably need to think carefully about the messaging and expectations around them.

                                                                                                                              1. 2

                                                                                                                                I can understand it from an administrative and marketing standpoint; it’s just a shame that we, society, can’t seem to have the option that protects privacy and also attracts the masses (who are not aware of the privacy threats, or see the convenience as a worthwhile tradeoff). Instead, someone such as myself feels put off and I can’t help be echo the schism it creates noted by the Drew Devault in Please don’t use Discord for FOSS projects . It’s the reason I suggest at a minimum listing unofficial bastions as a compromise.

                                                                                                                                1. 1

                                                                                                                                  It’s just too much work I’m afraid. If we had more money we could, but right now I’m stretched thin.

                                                                                                                            2. 1

                                                                                                                              As someone who works on community and chat systems I would love to hear which anti-harassment and anti-abuse features are most important to a community like yours.

                                                                                                                          1. 18

                                                                                                                            Todd said, “But that’s all in their heads!” and I agreed. Overwhelmed is a feeling, not a physical problem.

                                                                                                                            This. One thing I’ve learned from dealing with neurodivergent people, including myself, is that just because something only exists in your head doesn’t mean it’s not real. This applies to neurotypical people too! It’s just less noticeable because people are used to dealing with it a certain way. Once you accept this as a fact, no matter how silly or trivial something might look from the outside of someone’s head, then you can start actually addressing the problem.

                                                                                                                            1. 7

                                                                                                                              It’s just less noticeable

                                                                                                                              Is this even true? “it’s all in your head” is massively dismissive, no matter who you are. In fact, everything is “in your head” since we can’t even prove the physical world exists! What is in a person’s head is arguably the most real and the most noticeable to them

                                                                                                                              1. 3

                                                                                                                                is that just because something only exists in your head doesn’t mean it’s not real.

                                                                                                                                This line of thinking as written I believe to be wrong and to be a large part of the issues we have as a civilization (arguably, species).

                                                                                                                                An amended version I’m 100% fine with would be:

                                                                                                                                is that just because something only exists in your head doesn’t mean it’s not real to you.

                                                                                                                                It’s incredibly important to have empathy and a theory of mind that allows for interacting with irrational and insane people. It’s incredibly problematic to let others essentially dictate reality on their own whims, especially when there are more than one persons involved.

                                                                                                                                Sometimes, there is objectively a problem. Sometimes, there is a misinterpretation of an objective reality. Sometimes, the problem is rooted in an interpretation of nonreality. The techniques for addressing problems in these different categories are severely limited if we must take everything as actual reality–if you’ve ever dealt with a narcissist, schizophrenic, or borderline person, this should be obvious. For example, many of the successful techniques for dealing with anxiety require that you do not assume the brain/self has a correct view of reality.

                                                                                                                                If you’re ever in a management position, there are many cases where you’ll have to deal (as the author’s friend did) with folks who “feel” a certain way that is incongruous with facts on the ground. Burnout is one such circumstance, imposter syndrome another, plain old grudges and paranoia a third. You cannot effectively manage people if you automatically assume that what they’re telling you is the unvarnished truth–you can only assume that they believe those things to be true (and in pathological cases, you can’t even reliably assume that).

                                                                                                                              1. 7

                                                                                                                                Emacs, but not trapped in the ‘70s. A system – a holistic system, top to bottom, UI to drivers – that is live, inspectable, plastic, and fun. Let me explore my computer. I don’t care about security beyond the most basic, “don’t let unknown folks do things” extent. I don’t care about reliving the multi-user systems of my youth – I want my device to be mine. I don’t care about how easily it interacts with the rest of creation.

                                                                                                                                I want a UI at least as polished and sophisticated as the classic Mac OS, but not based on the world of scare storage and slow machines.

                                                                                                                                1. 3

                                                                                                                                  I am afraid that if you want sanity in drivers, what you dream of is not just reasonable software, but also hardware not actively hostile to reason.

                                                                                                                                  1. 3

                                                                                                                                    Have you tried Smalltalk?

                                                                                                                                    1. 1

                                                                                                                                      It sure sounds like SmallTalk, doesn’t it? I’ll have to give it a shot.

                                                                                                                                  1. 19

                                                                                                                                    A linker is something of a special case as it’s mostly used in a way where the business use is not hindered by (A)GPL in any way. Typically AGPL is great at getting companies to consider paying for a license.

                                                                                                                                    IANAL, but I think the author is making the right decision. I love FLOSS but I just don’t know of a FLOSS license that would work for a case like this.

                                                                                                                                    Should there be a more copylefty license that worked in a case like this? Is it even reasonably possible?

                                                                                                                                    1. 11

                                                                                                                                      AGPL is also exceptionally good at making companies not consider the software at all, eg Google: https://opensource.google/documentation/reference/using/agpl-policy

                                                                                                                                      1. 18

                                                                                                                                        Yes, that’s the point. Google, like many others, will pay for an alternative license if they want to use the project:

                                                                                                                                        In some cases, we may have alternative licenses available for AGPL licensed code.

                                                                                                                                        Of course the software needs to be exceptionally good to be considered exceptionally. I believe mold is.

                                                                                                                                        1. 4

                                                                                                                                          Google, like many others, will pay for an alternative license if they want to use the project

                                                                                                                                          Are there actually many cases of companies like Google paying for alternative licenses? When I was working at Google, it was rare and cases I saw were old legacy projects that nobody wanted to touch. For new projects, it was extremely uphill to get start procurement for a private licensing agreement. In practice, it’s often easier to just rewrite the [subset of the needed] software from scratch.

                                                                                                                                        2. 3

                                                                                                                                          That’s why the typical practice is for the software owner to license it to the business under a commercial license for a fee.

                                                                                                                                        3. 3

                                                                                                                                          Yes, if you’re trying to get someone to “pay for a license” you’re obviously not interested in open source, so I guess going nonfree is no surprise…

                                                                                                                                          1. 9

                                                                                                                                            That’s not obvious at all, plenty of people in open source sell licenses for their software, e.g. Red Hat Enterprise Linux, Oracle.

                                                                                                                                            1. 2

                                                                                                                                              I wouldn’t call either of RHEL or Oracle particularly “interested in open source”. They both do release some open source stuff, and also some not, and are also so huge and have such a mix of business models to go along with it that it’s not really comparable here.

                                                                                                                                              That said, I was replying to my parent comment, not to the post itself. The mold maintainer here does not seem to have been attempting to sell licenses before now, and is considering it for the future (and looking at BSL so the code is still open source in the end).

                                                                                                                                              1. 6

                                                                                                                                                Well, RH sponsors development of a ton of things, from systemd to podman, so that would be an incorrect assumption. Also Oracle sponsors OpenJDK and MySQL so again, incorrect.

                                                                                                                                                For the mold author, as someone else said, AGPL is a difficult license because of the software itself. So can’t blame them for looking for other options.

                                                                                                                                                1. 1

                                                                                                                                                  For what it’s worth, they’ve explicitly been trying to sell licenses to businesses since at least May: https://github.com/rui314/mold/commit/9fbd71ec6bb315c6fd4bfefbfcde821a4737b9e0

                                                                                                                                                  1. 1

                                                                                                                                                    Interesting that in the comments there they had the idea to maybe weaken to an MIT license but are now considering the opposite.

                                                                                                                                            2. 1

                                                                                                                                              I’m not well versed in the legal world but how does it work with GPL? Can one build non-GPL/prioprietary software using GPL compiler (eg. GCC) and link it with GPL linker (eg. GNU ld) and keep their license? Is it because of the GPL exception? AGPL licensed linker does not have this kind of exception so special license grant is required?

                                                                                                                                              1. 1

                                                                                                                                                The GCC runtime library exception is required because runtime support code is copied into the final binary. The same is not true of a linker.

                                                                                                                                                1. 1

                                                                                                                                                  So if GNU ld is used for linking, project must use GPL? Assuming its binary is distributed.

                                                                                                                                                  1. 6

                                                                                                                                                    No, the linker does not need the license exception. GCC both inserts fragments of code (for example for things like population count or count leading bit on architectures without instructions for these operations) and also inserts calls to and links several libraries that are part of the GCC codebase. As such, without the exception, everything spat out by GCC would need to comply with the GPL (it would not be GPL’d, it would simply need to convey the same rights as the GPL). In contrast, the linker just copies and pastes things from the input into the output and executes a small amount of code from the inputs to resolve relocations. It does not embed any of itself in the output and so it does not need the exception (in the same way EMACS does not need a GPL exception to avoid everything that you write in it from being GPL’d).

                                                                                                                                            1. 7

                                                                                                                                              Thinking locally, the author should do what’s needed for their project to succeed. I’m sympathetic.

                                                                                                                                              Thinking globally, I guess this is posted to Lobsters to discuss the larger philosophical issues? Okay, what would the world look like if the entire toolchain used by the author for their project was licenced as BSL (and always had been). Linux, gcc/clang, vi/emacs, etc. Or should all those other projects also now move to BSL, since they are clearly valuable to large parts of the software industry? What are the consequences for society? Do things get better or worse? If Open Source had been defined as “BSL” back in the day, then would the open source landscape today have even more and better choices, with better funding and salaries for developers, or would open source today be a wasteland?

                                                                                                                                              1. 13

                                                                                                                                                While I have mixed feelings about the BSL, and it’s not clear what their plan is int this specific case, it’s also important to note that BSL is an actual attempt at a compromise, unlike Commons Clause or other nonfree licenses. The code ends up as open source eventually, which is a big step up from some things they might be considering.

                                                                                                                                                1. 5

                                                                                                                                                  I am almost completely sure if BSL were more popular back in the day, we would have even more and better choices, with better funding and salaries for developers.

                                                                                                                                                  1. 4

                                                                                                                                                    I think you’re posing a wrong question. BSL and alike are not the enemy of Open Source. After all Open Source was a “business-friendly” response to Free Software. Like Free Software was a response to proprietary software. Each solved its own problem.

                                                                                                                                                    Free Software tried to make software accessible. And it succeeded. However, there was a problem. Business was reluctant to use that free software because of its viral lienes. No one knew what to make of it and tried to stay on the safe side. So FS was left mostly to hobbyists for the time.

                                                                                                                                                    Open Source came along to directly address adoption. By being more permssive OS licenses allowed the whole idea to spread beyond idiologic users and researchers. And OS succeeded.

                                                                                                                                                    However, now we have another problem. software widely adopted and used but there’s a lack of resources on the mantenance/development side of it. Too many requests from users, too much demand of developers’ time. BSL is an obvious thing to try. It’s hard to tell whether it will solve the issue but at least it’s a capitalist solution to a capitalist problem so there’s hope.

                                                                                                                                                    Now, let’s take your proposed scenario. I think it’d be just different. Maybe slightly better. The hole FOSS boom would’ve probably took a little longer as there would be some financial pressure agains software adoption but otherwise nothing would’ve changed much. There still would’ve been great demand for software. However, maintainers would’ve been in a much healthier position. And the whole idea of paying for software would be a little bit more acceptable. We probably wouldn’t see as many rants denouncing great software for authors wanting to pay their bills.

                                                                                                                                                    Just to reiterate. BSL is not an enemy of FOSS. Facebook would still push React under Open Source License, as would Google push Kubernetes and so on. Corporations are getting completely different thing out of it and for them Open Source licenses are as good as it can get.

                                                                                                                                                    1. 5

                                                                                                                                                      Free Software tried to make software accessible. And it succeeded. However, there was a problem. Business was reluctant to use that free software because of its viral lienes. No one knew what to make of it and tried to stay on the safe side. So FS was left mostly to hobbyists for the time.

                                                                                                                                                      I think this oversimplifies the history of these two camps to the point of false abstraction. The free software movement was never interested in business use of free software; they only cared about the agency of individual users. GNU was born out of a situation where commercial software was already taking over, commercial software will always be fine.

                                                                                                                                                      Open Source came along to directly address adoption. By being more permssive OS licenses allowed the whole idea to spread beyond idiologic users and researchers. And OS succeeded.

                                                                                                                                                      You make this sound as if free software underwent a licensing change to increase adoption, but this is not the case. No one can dispute that the Linux kernel is probably the most successful free software project in history. It remains GPL licensed. This has not changed since it’s inception.

                                                                                                                                                      The commercial adoption came first, and then more permissive licenses became more fashionable as using free software in business became normal. If permissiveness of licensing was the core issue, then surely a BSD derivative would have succeeded over Linux.

                                                                                                                                                      Open Source came about by a faction of people who wanted to commercialize free software. The new language was a rhetorical change intended to assuage the fears business had about viral licenses and open development. It was an attempt to sever the material differences in how software was developed under free licenses from the moralism of the FSF.

                                                                                                                                                    2. 2

                                                                                                                                                      I’m not sure - Ghostscript did this before with its dual license scheme. It makes one think twice before contributing - because now you’re contributing for free to something that makes money for someone else. It would also typically require you to sign a copyright assignment agreement so that the original author can legally continue doing what they are doing. For the original author(s), it also makes the decision to add someone to the core team more difficult, assuming it means they would now have to pay them. And I would expect so, because who would willingly spend more of their free time and work to fatten another man’s wallet?

                                                                                                                                                      On the whole, I don’t think open source would be a wasteland, per se. On one hand, it would slow down contributions, so maybe we wouldn’t be where we were today and have a lot less choice and products that are a lot less polished. On the other hand, it might mean people got paid and didn’t burn out so easily, and perhaps give them the opportunity to work on free software full-time, so like you said it might also be that we’d have more and better choices now, and more software would be source-available (so basically achieving the FSF’s goals, if in a somewhat diluted form)

                                                                                                                                                      1. 6

                                                                                                                                                        It makes one think twice before contributing - because now you’re contributing for free to something that makes money for someone else.

                                                                                                                                                        I don’t mind that they make money, I mind the asymmetry: they can make money in a way that I cannot, from a project that we both contribute to. This principle of fairness is critical in most F/OSS licenses: if you and I collaborate on something then we both have the same rights to the result. There are a few exceptions but they tend to be unpopular.

                                                                                                                                                        1. 6

                                                                                                                                                          This issue becomes especially relevant when the original project becomes abandoned and someone has to start a fork to keep it alive. There is no clause in BSL that covers that. If the original authors do not explicitly re-license the code (which they may not even be able to do, in case of a hostile takeover, for example), then forking isn’t a practical possibility at all.

                                                                                                                                                          I suppose it may even make companies that have popular products under BSL attractive targets for takeovers, since communities cannot do anything to keep the open-source version alive — everyone who wants to keep using the software has to buy it from the new owner on their new terms.

                                                                                                                                                          1. 4

                                                                                                                                                            That’s an excellent point. From a corporate perspective, the ability to create a second source if none exists is one of the key values of F/OSS. If a company takes a project in a direction that you, as a customer, don’t like then you can pay any other company that is willing to bid to maintain a fork that goes in the direction that you want. That’s a last resort, but the fact that it’s possible is generally enough of an incentive to stop the maintainers going too far off the rails. With dual licensing, you lose this. You may be happy to pay for version X, but if version X + N drops features that you depend on, has show-stopper bugs for you, or whatever then you can’t just pay someone to fix the open version you need to negotiate with the maintainers to add the features that you need. That makes it much higher risk than a real F/OSS license that is sufficiently permissive for whatever you want to be able to do with it.

                                                                                                                                                            1. 2

                                                                                                                                                              With dual licensing, you lose this

                                                                                                                                                              I think we need a specific term for BSL-like licenses. “Dual licensing” is broad and includes situations like licensing the code under two different FOSS licenses (like GPL + MPL), which isn’t harmful at all. Any dual licensing where one of the licenses is FOSS and both are applied at the same time also doesn’t prevent forks.

                                                                                                                                                              Maybe “delayed dual licensing” could be a good term.

                                                                                                                                                              1. 1

                                                                                                                                                                Sorry, I was thinking of the GhostScript case, where it was GPL’d or proprietary.

                                                                                                                                                                A lot of F/OSS dual licensing is probably not legally binding, because one license grants a superset of the rights of the other. This means that the ‘reasonable person’ test that the court would use will assume no one would ever choose to receive fewer rights and so the licenses are equivalent to the most permissive one. In cases where the licenses are incompatible (e.g. GPL + CDDL), then it’s far more complex and you probably need to take it to court to figure out what it actually means.

                                                                                                                                                    1. 33

                                                                                                                                                      All the power to OP. One thing I notice is how seemingly divorced they are to “sales”, though. If you are running a one person startup, even if it’s free software, you will need to be doing sales in some way or another.

                                                                                                                                                      This release disclaimer is just another form of sales — a bit clumsy one TBH — and it should link to a call to action on the startup website.

                                                                                                                                                      Conference talks, reaching out to the folks at BigCo et al. that can actually move the lever to get your free software to get sponsored.

                                                                                                                                                      1. 21

                                                                                                                                                        This. Changing the copyright license isn’t a magic replacement for having a solid sales process.

                                                                                                                                                        1. 11

                                                                                                                                                          Which is a shame. Sales is a completely different type of activity to development. And yet another time/effort sink cutting into making the actual product.

                                                                                                                                                          I’m not saying it’s useless. Just that it’s a shame that being good at one thing is not enough and actual success requires learning an entirely different profession.

                                                                                                                                                          1. 5

                                                                                                                                                            I guess it’s a matter of semantics. You can “sell” your project in more ways than traditional sales.

                                                                                                                                                            But if you’re making something you’d like other people to use and give you money for, you’re going to be doing some kind of sales along the way — or associate with someone who will.

                                                                                                                                                            1. 4

                                                                                                                                                              Most professions that involve launching your own work require multiple competences and skills to be in place for it to succeed. Making a profit from a product is the final step in a relay race by multiple professionals.

                                                                                                                                                              1. 2

                                                                                                                                                                But here’s the problem from the payers side: I have money for 5 good things to fund and there’s 100 out there. How should I pick?

                                                                                                                                                                Sales is also not an entirely new profession, many things can be taken over. E.g. successfully funded projects give regular progress updates and label them as an accountability measure. Just enough sales is enough in a place where no one does any sales.

                                                                                                                                                                E.g. one basic that can be immediately applied: do pitch your ideas end of October/Beginning November. That’s when people do their budgets for next year. And if $bigco does find money in the cracks for something they find useful, it’s okay to ask for 5-6 figures.

                                                                                                                                                                1. 1

                                                                                                                                                                  But here’s the problem from the payers side: I have money for 5 good things to fund and there’s 100 out there. How should I pick?

                                                                                                                                                                  Start by picking all 100. Then designate the 5 you consider most important, and allocate a percentage to them, such that the sum of those percentages is less than 100. Then whatever the remainder is gets split among the remaining 95. I’m just brainstorming here. We’re a community of highly intelligent folk, and I’m sure someone could come up with something much better.

                                                                                                                                                                  Nobody wants to sit down and mark all of the things they use. The distros can help here. For instance, a program could scan your package database and give you a list of possible funding candidates to choose from.

                                                                                                                                                                  It would be nice to have all of this decentralized. Maybe the way to do it would be with some sort of proof-of-stake cryptocurrency. A project interested in funding publishes an address where it can receive funds. I’m talking out of my ass here, because I’m a fairly anti-crypto kind of person. What we really need is the equivalent of the email address for making frictionless payments to individuals and small organizations.

                                                                                                                                                                  What I’m getting at with all this stream of consciousness brainstorming is that I think it’s a solvable problem.

                                                                                                                                                                  The real longterm solution is UBI, but we aren’t there yet, and we may not be in my lifetime. So a stopgap is needed.

                                                                                                                                                                  1. 2

                                                                                                                                                                    Even companies that want to support open source aren’t going to do micro-donations to a hundred projects. Companies are going to make larger donations to one flagship thing they use, or maybe to a handful of them, and call it a day.

                                                                                                                                                                    And every service that’s sprung up to try to be an intermediary has had massive red flags, at least in my opinion. Way too many of them run on the model of:

                                                                                                                                                                    • Collect donations “on behalf of” anything with a public GitHub URL
                                                                                                                                                                    • Don’t mention that those projects have never heard of this “helpful” service
                                                                                                                                                                    • Require projects to come explicitly sign up in order to receive money collected “on their behalf”
                                                                                                                                                                    • “Redistribute” donations targeted for any projects that don’t come sign up within a certain short window of time

                                                                                                                                                                    People have been reviving this “helpful” service about once every eighteen months for the past decade or so. I’ve repeatedly pointed out how they’re the opposite of helpful and may actually cause massive tax and legal problems for the developers and projects they’re supposed to “help”, but for some reason the people running these services never seem to care much about that.

                                                                                                                                                            1. 1

                                                                                                                                                              There are lots of them, but I actually thing for this use case distributed is less flexible than federated. A federated protocol you can host your own, or not, and host it anywhere. Distributed you have to host your own, and often in up doing it on an end device like a mobile phone where it eats the battery.

                                                                                                                                                              1. 1

                                                                                                                                                                Are there any examples of battery-friendly P2P protocols?

                                                                                                                                                                Perhaps protocols could have two tiers of P2P nodes: leaf nodes and supernodes (name stolen from Skype). Phones would be leaf nodes, and would not require constant connectivity. Perhaps the leaf-supernode trust relationship could be assured via Verifiable Data Structures ( https://security.googleblog.com/2021/07/verifiable-design-in-modern-systems.html ).

                                                                                                                                                                Federated protocols (Mastondon, XMPP, Email) are just that, but without the cryptographic assurance between leaf and supernode. In practice that might be sufficient.

                                                                                                                                                                1. 1

                                                                                                                                                                  Yeah, exactly. Before I got to your last bit I was thinking “you’re just describing federation” heh