Threads for peterbourgon

  1. 16

    To summarise as fairly as I can: the author sees people trying to write correct code via unit testing. They are puzzled by this because Erlang isn’t supposed to be correct, it’s supposed to have bugs and then do some work to isolate those bugs, to be able to crash but the whole system keeps mostly running in the face of that crash.

    But like… wouldn’t you like to do both? Not have the bugs, and also safely crash when you have the bugs? I can’t tell if this is meant to be ironic or not.

    1. 11

      The goal for the ‘let it crash’ mindset isn’t to ship buggy code, it’s to acknowledge that even formally verified code is not bug free (your specification can never be complete) and your system should aim to be correct in the presence of failures. Testing is important in Erlang, but testing should be of the form of killing processes at surprising times and ensuring that the system recovers and does not lose data. This gives you a far more reliable system than a quest to eliminate bugs via unit tests. You have a limited budget for testing, you should spend it where it will deliver the most impact.

      1. 18

        Randomly crashing processes isn’t going to help me if I’m testing my deflate implementation or whatever. Regular unit tests are still very useful in Erlang for checking that functions do what they ought to.

        1. 4

          I’m familiar with Erlang and with let-it-crash, but that’s not the claim the article is making. Here it is blow-by-blow:

          1. I see people writing a lot about unit testing in Erlang
          2. Do we really need all that?
          3. Erlang has some testing stuff
          4. But there are cases that it can’t catch, and neither would a type system*
          5. Erlang’s history includes environments that needed very high uptimes
          6. To do that, Erlang has a way to continue in the face of crashes
          7. If you write “clear code with modestly-sized routines” you don’t need tests
          8. Some specific classes of bugs related to mutability aren’t possible
          9. Erlang has supervisors that can restart and log buggy code
          10. ∴ “some Erlangutans [are] missing the point of using Erlang”

          The only substantive thing there that could possibly mitigate the need for individual functions to be correct is the “clear code with modestly-sized routines”. Okay, yeah, if you never write wrong code then you’ll never have wrong code, that’s not unique to Erlang. But nothing here obviates the need for correctness. Let-it-crash allows your system that does 10 things to keep doing 9 of them while it’s broken. But it doesn’t make the other one not broken. It doesn’t make that function correct. I’m not even a TDD weenie myself but the idea that let-it-crash is in any way related to correctness, whatever form that might take, strikes me as absurd.

          Let-it-crash is very situational. It’s good to allow a complex system to limp along in the face of temporary external problems or problems only involving parts of the system. But “the point” of Erlang isn’t that absolves you from writing correct code with a magical “On Error Resume Next” that still builds your widgets even if you close the exit hatch before you eject them from the chute. Let-it-crash lets one of your two widget-builders keep running after one trips the breaker, which is great and is actually the “point of using Erlang”. But if you don’t fix that bug you’ll still never build a single widget.

          *: that particular case can be caught by some type systems like affine or linear types. I don’t claim that Erlang would be improved by those, just some fun trivia

          1. 1

            How can you be correct with unforeseen failures? Seems like the general opinion around this is if an unplanned error happens, almost all bets are off and you can’t trust anything later to be correct so you should just crash and get a person to review the problem.

            1. 1

              How should we distinguish between foreseeable and unforeseeable failures? If I write a program that tries to read a file whose path is provided by user input, is it not foreseeable that this file may not exist? Under what conditions is it appropriate for the entire program to crash in response to this error? Under what conditions is it my responsibility as a programmer to explicitly deal with this error? What about errors during file reads caused by device errors? Or errors during file descriptor close or flush or sync syscalls? Or errors when connecting to a remote API? Or during network transfers? Or when communicating with a database?

              If you want to define crashworthy errors as whatever the programmer says are crashworthy errors, okay! But then what does crashing mean? If you’re serving a request in an Erlang actor and you hit a syscall error then it’s pretty OK to “crash” that actor because actors are request-scoped and the crash just terminates that single request. But if you’re serving a request in a concurrent e.g. Rust program and that request hits a syscall error then you definitely shouldn’t “crash” the entire process, right? You can terminate the request, sure, but that’s not really what anybody understands by “crash” terminology.

              1. 3

                At my (former) job, I used Lua to process incoming SIP messages. It uses Lua coroutines [1] to handle each transaction. If some unexpected thing happens (like a nil reference missed in testing due to unexpected input for instance) only that coroutine “crashes”—that is, ceases to run. This is caught by the framework I’m using and logged. The “crash” does not affect the rest of the program. In my experience, most of the crashes were due to mishandling of input, or rather, a miscommunication about what, exactly, we could expect from the Oligarchic Cell Phone Company that was our customer [2]. We don’t expect crashes, but sometimes we do overlook some conditions (as I’m fond of saying, “bugs are caused by an inattention to detail.”).

                While Lua isn’t Erlang, I think the overall effect is the same—a crashed thread does not impinge on the rest of the program. Perhaps a better distinction is “let it soft crash [3].”

                [1] Think threads, but not pthreads threads, but Lua specific ones cooperatively mutlitasked.

                [2] Our service is only for the North American Numbering Plan. We did not expect to receive international phone numbers at all (and weren’t for the non-SIP legacy interface).

                [3] Where a “soft crash” just stops the processing and a diagnostic log is issued.

                1. 1

                  Lua coroutine . . . [crashes are] caught by the framework I’m using

                  Great! Good! My point here is not to assert an absolute, or to deny your specific experience in any way. My point here is to say that the general notion of “crashing” usually does not describe the act of terminating entities under the management of an eg Erlang OTP, or your Lua framework, but instead usually describes the act of terminating entire OS processes.

                  a crashed thread

                  I’m trying to communicate that most people do not generally understand threads as things that can “crash”. Threads die or terminate, processes crash. The difference in terminology is my entire point.

                  1. 1

                    Then what would you call it?

                    I have a Lua script. I try running it, Lua spits out “attempt to index a nil value (global ‘x’)” and ends the program. That is, if I understand your nomenclature, a “crash.” Yet if I wrap that same code in a coroutine, the coroutines fails to finish, yet the script continues. What did the coroutine do? Just terminate? I personally consider it a “crash.”

                    And what does it mean for a “process” to crash? On Unix, the system continues running. Yet on MS-DOS, such a “crash” will usually “crash” the computer. Yet both are crashes. Why can a “process” running under a protected domain (Unix) crash, and yet, threads cannot? I think the participants here are using a broader definition of “crash” than you personally like.

                    1.  

                      I have a Lua script. I try running it, Lua spits out “attempt to index a nil value (global ‘x’)” and ends the program. That is, if I understand your nomenclature, a “crash.” Yet if I wrap that same code in a coroutine, the coroutines fails to finish, yet the script continues.

                      If you can write a bit of code that runs (more or less) equivalently as its own OS process (scenario 1) or as a coroutine among other coroutines in a single shared OS process without modification (scenario 2) then whatever is orchestrating coroutines in the second scenario is necessarily enforcing isolation boundaries that are functionally equivalent to the OS threads in the first scenario.

                      What did the coroutine do? Just terminate? I personally consider it a “crash.”

                      If that code fails in scenario 1, is that a crash? IMO yes.

                      If that code fails in scenario 2, is that a crash? IMO no.

                      Why can a “process” running under a protected domain (Unix) crash, and yet, threads cannot? I think the participants here are using a broader definition of “crash” than you personally like.

                      My understanding of “crash”, or yours, isn’t important to me, what I care about is what the average developer understands when they see that term, absent any other qualifications. I don’t think most developers consider a thread termination to be a crash, and I think if you want to say “crash” to mean something like that you need to qualify it. That’s my only point. Could be wrong.

                2. 2

                  It’s worth pointing out that in the Erlang world, it’s not the entire program that crashes. Like, ever. Instead, whatever individual process that ran into the error is the piece that fails. In the context of a program loading a file, it’s likely that just the process responsible for loading the file would die. Which makes perfect sense, since it’s no longer a meaningful operation to continue loading a file that we can’t open.

                  The elegance of the “let it crash” mentality is that it lets you handle foreseeable, unforeseeable, and foreseeable-but-too-unlikely-to-care-about failures in the exact same way. Like sure, you could check to make sure a file exists, but what if it’s corrupted? Or someone takes a sledgehammer to the hard drive while you’re in the midst of reading the file? There’s an infinite number of failure states for a given operation, and for a lot of them there isn’t a meaningful resolution within the operation itself.

                  1. 1

                    It’s well-understood that “crashing” in Erlang doesn’t mean crashing the entire program. The problem is that “crashing” in basically any context except Erlang does mean crashing the entire program.

          1. 30

            I think size of the program, and the team maintaining it, is an important factor in the static vs dynamic discussion.

            I’m in the “I don’t make type errors, and if I do, I can shake them out with a few tests” camp for as long as I can comprehend what’s going on in the codebase.

            But when the code grows to the point I can no longer remember where everything is, e.g. I can’t find all callers of a function, dynamism starts to become a burden. When I need to change a field of a struct, I have to find all uses of the field, because every missed one is a bug that’s going to show up later. At some point that becomes hard to grep for, and even impossible to account for in reflection-like code. It can degrade to a bug whack-a-mole, promote more and more defensive programming patterns, and eventually fear of change.

            I’ve had good experiences with gradually-typed languages. They stretch this “size limit” of a program a lot, while still allowing use of duck typing where it helps, without bringing complexity of generic type systems.

            1. 10

              “Dynamic typing falls apart with large team/large codebase” is one of those cliché arguments that doesn’t really contribute usefully, though.

              Also your presentation of it has multiple issues:

              • Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.
              • Tooling for dynamically-typed languages has come a long way in the decades since this argument was first raised. You can just get an IDE and tell it to track down and rename references for you. And if your complaint is that it’s harder/impossible to do through “reflection-like code”, well, people can write metaprogram-y reflection stuff in statically-typed languages too.
              • Ultimately, if your codebase has lots of functions or methods that are called from huge numbers of disparate places, to such a degree that you can’t safely work with it without an IDE doing full static analysis to track them all for you, that’s a code smell in any language, in any typing discipline.
              1. 18

                Static languages can verify all metaprogramming is type correct. IDE heuristics can not. In Rust you can write a macro and the compiler will expand and type check it. That kind of stuff is impossible in dynamic languages.

                1. 8

                  Static languages can verify all metaprogramming is type correct.

                  This is probably going to get off-topic into arguing about the exact definition of “statically-typed”, but: I think that if you venture outside of languages like Rust (which seem to deliberately limit metaprogramming features precisely to be able to provide guarantees about the subset they expose), you’lll find that several languages’ guarantees about ahead-of-time correctness checks start being relaxed when using metaprogramming, runtime code loading, and other “dynamic-style” features. Java, for example, cannot actually make guarantees as strong as you seem to want, and for this among other reasons the JVM itself is sometimes referred to as the world’s most advanced dynamically-typed language runtime.

                  There also are plenty of things that seem simple but that you basically can’t do correctly in statically-typed languages without completely giving up on the type system. Truly generic JSON parsers, for example. Sure, you can parse JSON in a statically-typed language, but you either have to tightly couple your program to the specific structures you’ve planned in advance to handle (and throw runtime errors if you receive anything else), or parse into values of such ultra-generic “JSON object” types that the compiler and type system no longer are any help to you, and you’re effectively writing dynamically-typed code.

                  1. 3

                    Dynlangs are definitely better for data that isn’t structured as well.

                    C#’s dynamic keyword feels like a perfect fit for this situation without having to give up static typing everywhere else. Hejlsberg is ahead of the curve, per usual.

                    1. 3

                      for this among other reasons the JVM itself is sometimes referred to as the world’s most advanced dynamically-typed language runtime

                      Aren’t runtimes always “dynamically typed”? What does it mean for a runtime to be “statically typed”?

                      or parse into values of such ultra-generic “JSON object” types that the compiler and type system no longer are any help to you, and you’re effectively writing dynamically-typed code.

                      It sounds like you’re arguing that the worst case for static type systems is equivalent to the best case for dynamic type systems, which doesn’t seem like a ringing endorsement for dynamic type systems. That said, I don’t even think this is true for this JSON-parsing example, because you could conceive of a generic JSON parser that has different unmarshaling strategies (strict, permissive, etc). Further, as static type systems are adopted more widely, this sort of poorly-structured data becomes rarer.

                      1. 5

                        Aren’t runtimes always “dynamically typed”?

                        Some more so than others. Rust, for all its complexity as a language, is mostly shoving that complexity onto the compiler in hopes of keeping the runtime relatively simple and fast, because the runtime doesn’t have to do quite as much work when it trusts that there are classes of things the compiler simply prevents in advance (the runtime still does some work, of course, just not as much, which is the point).

                        But a language like Java, with runtime code loading and code injection, runtime reflection and introspection, runtime creation of a wide variety of things, etc. etc. does not get to trust the compiler as much and has to spend some runtime cycles on type-checking to ensure no rules are being broken (and it’s not terribly hard to deliberately write Java programs that will crash with runtime type errors, if you want to).

                        That said, I don’t even think this is true for this JSON-parsing example, because you could conceive of a generic JSON parser that has different unmarshaling strategies (strict, permissive, etc).

                        If you want truly generic parsing, you’re stuck doing things that the compiler can’t really help you with. I’ve seen even people who are quite adept at Haskell give up and effectively build a little subset of the program where everything is of a single JSON type, which is close enough to being dynamically typed as makes no difference.

                        Further, as static type systems are adopted more widely, this sort of poorly-structured data becomes rarer.

                        My experience of having done backend web development across multiple decades is that poorly-structured data isn’t going away anytime soon, and any strategy which relies on wishing poorly-structured data out of existence is going to fail.

                        1. 2

                          Aren’t runtimes always “dynamically typed”?

                          If you eschew these needlessly binary categories of static vs dynamic and see everything on a scale of dynamism then I think you’ll agree that runtimes are scattered across that spectrum. Many even shift around on that spectrum over time. For example, if you look at the history of JSR 292 for adding invokedynamic to the JVM you’ll find a lot of cases where the JVM used to be a lot less dynamically typed than it is today.

                        2. 2

                          There’s no reason you can’t parse a set of known JSON fields into static members and throw the rest into an ultra-generic JSON object.

                          1. 3

                            Those are the options I said are available, yes.

                            1. 1

                              I mean, you can do both at once for the same value, getting the benefits of both.

                      2. 7

                        Fail is too harsh. Unless you’re writing some rocket navigation system, a project is not going to outright fail because of software defects. Run-time type errors merely add to other bugs that you will need to fix, and I argue that bugs caused by runtime type errors are less of a problem in small programs.

                        I don’t know of any robust tooling for refactoring large JavaScript projects. Of course most languages have some type-system escape hatches, but I expect languages like JS to use hard-to-analyze type-erasing constructs much more often.

                        I disagree that having callers beyond your comprehension is automatically a code smell. It’s a natural state of things for libraries, for example. Ideally libraries should have a stable API and never change it, but it’s not always that easy, especially for internal libraries and reusable core pieces of large projects that may need to evolve with the project.

                        It’s not just about IDEs. Compilation will also track down all type errors for you, regardless of where and when these errors happen. When working with teams, it may be someone else working on some other component. In this case the types are a way to communicate and coordinate with others.

                        You can make a mess in any language, but how easy is to make a mess varies between languages. Languages that prevent more errors will resist the mess for longer.

                        1. 2

                          I expect languages like JS to use hard-to-analyze type-erasing constructs much more often.

                          Why do you expect this?

                          I disagree that having callers beyond your comprehension is automatically a code smell.

                          Even if it’s an internal library, why don’t other internal codebases have a single clear integration point with it? And why does everything else need to have lots of knowledge of the library’s structure? This definitely is a code smell to me – the Law of Demeter, at least, is being violated somewhere, and probably other design principles too.

                          Languages that prevent more errors will resist the mess for longer.

                          This is veering off into another clichéd and well-trod argument (“static typing catches/prevents more bugs”). I’ll just point out that while proponents of static typing often seem to take it as a self-evident truth, actually demonstrating its truth empirically has turned out to be, at the very least, extremely difficult. Which is to say: nobody’s managed it, despite it being such an “obvious” fact, and everybody who’s tried has run into methodological problems, or failed to prove any sort of meaningful effect size, or both.

                          1. 1

                            Why do you expect this?

                            Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                            why don’t other internal codebases have a single clear integration point with it?

                            The comprehension problems I’m talking about that appear in large programs also have a curse of being hard to explain succinctly in a comment like this. This is very context-dependent, and for every small example it’s easy to say the problem is obvious, and a fix is easy. But in larger programs these problems are harder to spot, and changes required may be bigger. Maybe the code is a mess, maybe the tech debt was justified or maybe not. Maybe there are backwards-compat constraints, interoperability with something that you can’t change, legacy codebase nobody has time to refactor. Maybe a domain-specific problem that really needs to be handled in lots of places. Maybe code is weirdly-shaped for performance reasons.

                            The closest analogy I can think of is “Where’s Waldo?” game. If I show you a small Waldo picture, you’ll say the game is super easy, and obviously he’s right here. But the same problem in a large poster format is hard.

                            1. 4

                              Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                              You are once again assuming that statically-typed languages catch/prevent more errors, which I’ve already pointed out is a perilous assumption that nobody’s actually managed to prove rigorously (and not for lack of trying).

                              Also, the explanation you give still doesn’t really make sense. Go look at some typical Python code, for example – Python’s metaprogramming features are rarely used and their use tends to be discouraged, and easily >99% of all real-world Python code is just straightforward with no fancy dynamic tricks. People don’t choose dynamic typing because they intend to do those dynamic tricks all the time. They choose dynamic typing (in part) because having that tool in the toolbox, for the cases when you need it or it’s the quickest/most straightforward way to accomplish a task, is incredibly useful.

                              The comprehension problems I’m talking about that appear in large programs also have a curse of being hard to explain succinctly in a comment like this

                              Please assume that I’ve worked on large codebases maintained by many programmers, because I have.

                              And I’ve seen how they tend to grow into balls of spaghetti with strands of coupling running everywhere. Static typing certainly doesn’t prevent that, and I stand by my assertion that it’s a code smell when something is being called from so many disparate places that you struggle to keep track of them, because it is a code smell. And there are plenty of patterns for preventing it, none of which have to do with typing discipline, and which are well-known and well-understood (most commonly, wrapping an internal interface around a library and requiring all other consumers in the codebase to go through the wrapper, so that the consuming codebase controls the interface it sees and has onlyu a single point to update if the library changes).

                              1. 2

                                I’ve worked on large codebases maintained by many programmers, because I have. And I’ve seen how they tend to grow into balls of spaghetti with strands of coupling running everywhere. Static typing certainly doesn’t prevent that . . .

                                No, definitely not, agreed. But static typing definitely improves many/most dimensions of project maintainability, compared to dynamic typing. This isn’t really a controversial claim! Static typing simply moves a class of assertions out of the domain of unit tests and into the domain of the compiler. The question is only if the cost of those changes is greater or lesser than the benefits they provide. There’s an argument to be made for projects maintained by individuals, or projects with lifetimes of O(weeks) to O(months). But once you get to code that’s maintained by more than 1 person, over timespans of months or longer? The cost/benefit calculus just doesn’t leave any room for debate.

                                1. 2

                                  But static typing definitely improves many/most dimensions of project maintainability, compared to dynamic typing. This isn’t really a controversial claim!

                                  On the contrary, it’s a very controversial claim.

                                  Proponents of static typing like to just assert things like this without proof. But proof you must have, and thus far nobody has managed it – every attempt at a rigorous study to show the “obvious” benefits of static typing has failed. Typically, the ones that find the effect they wanted have methodological issues which invalidate their results, and the ones that have better methodology fail to find a significant effect.

                                  The cost/benefit calculus just doesn’t leave any room for debate.

                                  Again: prove it. WIth more than anecdata, because we both have anecdotes and that won’t settle anything.

                              2. 2

                                Because the flexibility is a benefit of dynamic languages. If you try to write code as-if it was strongly statically typed, you’re missing out on the convenience of writing these things “informally”, and you’re not getting compiler help to consistently stick to the rigid form.

                                I see most typing errors as self-inflicted wounds at this point. Don’t have time or patience for things that can be prevented by the compiler happening at runtime.

                                Dynlangs + webdev together is my kryptonite. If I had to do that all day I’d probably start looking for a new career. Just can’t deal with it.

                            2. 1

                              I don’t know of any robust tooling for refactoring large JavaScript projects

                              Following up on this specifically: I’m an Emacs guy, not really an IDE person, so I don’t know the landscape that well. But everybody I know who goes the IDE route in Python uses PyCharm, so I looked up the rest of the JetBrains product line, and sure enough they have an IDE for JavaScript/TypeScript which claims to support refactoring.

                              I assume it’s not the only such product out there.

                            3. 3

                              Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.

                              Yes, projects can fail for lots of reasons; no one is claiming that static typing will make a shitty idea commercially successful, for example :) But I do think static types help a lot within their narrow scope–keeping code maintainable, reducing bugs, preserving development velocity, etc. Of course, there’s no good empirical research on this, so we’re just going off of our collective experiences. 🤷‍♂️

                              1. 1

                                Large team/large codebase projects fail all the time regardless of typing discipline. Static typing doesn’t appear to have a better track record of success there.

                                I think it pretty much does, actually. Static typing moves an enormous class of invariants from opt-in runtime checks to mandatory compile-time checks. Statically typed languages in effect define and enforce a set of assertions that can be approximated by dynamically typed languages but never totally and equivalently guaranteed. There is a cost associated with this benefit, for sure, but that cost is basically line noise the moment your project spans more than a single developer, or extends beyond a non-trivial period of time.

                                1. 1

                                  I think it pretty much does, actually.

                                  As I said to your other comment along these lines: prove it. The literature is infamously full of people absolutely failing to find effects from static typing that would justify the kinds of claims you’re making.

                              2. 13

                                I always laugh when I see ruby code where the start of the method is a bunch of “raise unless foo.is_a? String”. The poor mans type checking all over the place really highlights how unsuitable these dynamic languages are for real world use.

                                1. 7

                                  To be fair, any use of is_a? in ruby is a code smell

                                  1. 12

                                    Sure, it’s also a pattern I have seen in every Ruby codebase I have ever worked with because the desire to know what types you are actually working with is somewhat important for code that works correctly.

                                    1. 5

                                      Yeah, the need for ruby devs is much larger than the supply of good ones or even ones good enough to train the others. I’ve seen whole large ruby codebases obviously written by Java and C++ devs who never got ruby mentoring. I expect this is an industry wide problem in many stacks

                                  2. 5

                                    You seem to just be trolling, but I’ll play along, I guess.

                                    I’ve seen a bit of Ruby, and a lot of Python and JavaScript, and I’ve never seen this except for code written by people who were coming from statically-typed languages and thought that was how everyone does dynamic typing. They usually get straightened out pretty quickly.

                                    Can you point to some examples of popular Ruby codebases which are written this way? Or any verifiable evidence for your claim that dynamic languages are “unsuitable… for real world use”?

                                    1. 5

                                      I’m not trolling at all. I’ve been a Rails dev for the last 7 years and seen the same thing at every company. I don’t work on any open source code so I can’t point you at anything.

                                      I quite like Rails but I’m of the opinion that the lack of static type checking is a serious deficiency. Updating Rails itself is an absolute nightmare task where even the official upgrade guide admits the only way to proceed is to have unit tests on every single part of the codebase because there is no way you can properly verify you have seen everything that needs to change. I’ve spent a large chunk of time spanning this whole year working towards updating from Rails 5.1 to 5.2. No one else dared attempt it before I joined because it’s so extremely risky.

                                      I love a lot of things about Rails and the everything included design but I don’t see a single benefit to lacking types. Personally I see TypeScript as taking over this space once the frameworks become a little more mature.

                                      1. 3

                                        You made a very specific assertion about how people write Ruby (lots of manual type-checking assertions). You should be able to back up that assertion with pointers to the public repositories of popular projects written in that style.

                                        1. 7

                                          I remembered hearing from my then-partner that Rails itself uses a lot of is_a?, and that seems true.

                                           if status.is_a?(Hash)
                                                  raise ArgumentError, etc...
                                          
                                          1. 3

                                            This is pretty misleading – a quick glance at some of the examples seems like many of them aren’t really checking argument types, and when they are, they’re often cases where a method accepts any of multiple types, and there’s branching logic to handle the different options.

                                            Which is something you’d also see in a statically-typed language with sum types.

                                            The proposition that this is a common idiom used solely as a replacement for static checking is thus stil unproved.

                                            1. 1

                                              Well yeah, and then there’s those that raise errors, or return some failure-signaling value.

                                              I don’t know what projects to look at since I don’t use Ruby, but I found some more in ruby/ruby.

                                          2. 6

                                            ill concur with GP: this is a fairly common pattern to see in ruby codebases.

                                            however, to be fair, it’sa pattern most often introduced after attending a talk by a static typing weenie…

                                      2. 4

                                        Do you also laugh when you see “assert(x > 0);” in typed languages?

                                        1. 6

                                          I would, but it would be a sad laugh because I’m using a type system that can’t express a non-zero integer.

                                          1. 2

                                            I would love to see broader adaptation of refinement types that let you statically guarantee properties like integer values being bound between specific values.

                                        2. 3

                                          I’m in the Type everything if it’s even kinda big camp now. There are too many things I need to think about during the day to remember the state and usage of every variable of every program I’ve ever written, used or inspected. Typings are rails for my logic. Typings are documentation. Types help my IDE help me. I will take every single shortcut I can when the timespan I or anyone else could be interacting with the code is longer than 10 minutes.

                                          Retracing steps is just so tedious and frustrating when you had it all in your head before. It just sucks. I just wanna build stuff, not fill my head with crap my computer can do.

                                          /rant

                                          1. 2

                                            I’m in the “I don’t make type errors, and if I do, I can shake them out with a few tests” camp for as long as I can comprehend what’s going on in the codebase.

                                            This is generally true for me, but writing tests or debugging stack traces makes for a slow iteration loop. A type error from a compiler usually contains better, more direct information so resolving these type errors is a lot faster. To the extent that I (a 15 year Pythonista) eventually began to prototype in Go.

                                            That said, the biggest advantage for me for a static type checker is that it penalizes a lot of the crappy dynamic code (even the stuff that is technically correct but impossible to maintain/extend over time). A static type system serves as “rails” for less scrupulous team members. Of course, a lot of these less-scrupulous developers perceive this friction as a problem with static type systems rather than a problem with the way they hacked together their code, but I think Mypy and TypeScript have persuaded many of these developers over time to the extent that static types are much less controversial in most dynamic language communities.

                                            Another big advantage is that your type documentation is always correct and precise (whereas docs in a dynamically typed language often go stale or simply describe something as “a file-like object” [does that mean it just has a read() method, or does it also need write(), close(), seek(), truncate(), etc?]). Further, because the type docs are precise, you can have thinks like https://pkg.go.dev complete with links to related types, even if those types are declared in another package, and you get all of this for free.

                                          1. 38

                                            This is a curious misunderstanding of what “let it crash” means.

                                            “Let it crash” is the observation that if something unexpected, maybe spurious happens in a distributed, ever-changing system, it’s a good approach to crash the unit of work in a structured way and have the supervisor of that unit figure apply a strategy. e.g. if you are connecting to a socket, doing some work and then close - and the connection doesn’t work, the task should crash. The supervisor could then retry 3 times and then propagate the error. It’s a philosophy for error handling and structuring errors in a concurrent system, at far more high level then the example in the code.

                                            That approach is very inspired from how init systems are doing things. Say, you run a cleanup task using your init system - it fails once because of a spurious connection error. The init system tries it 5 more times. If it fails 5 times, it then marks the task faulty. Same philosophy: your init system is a supervision system.

                                            “Let it crash” does not mean “your code can be faulty”, it encourages crashing as a strategy for handling of spurious, hard to predict errors.

                                            1. 26

                                              This is a curious misunderstanding of what “let it crash” means.

                                              Their understanding of “let it crash” is not your understanding of “let it crash”, and–regardless of the degree to which I prefer your explanation–classifying it as a misunderstanding when I believe they have more expertise in the Erlang ecosystem (7 years at least, judging solely from this blog) and idioms than you do is somewhat presumptuous.

                                              More pithily: please don’t rustsplain to the erlangutans. :P

                                              There are things I disagree with in the article–for example, I think that tests are helpful for refactoring and for things like numerical routines…here the author and I part ways. That said, perhaps the misunderstanding here is that you and author view bugs differently: the author says that “let it crash” covers all defects and failure conditions (including ones introduced by programmer negligence or bad luck or foresight) while you say it covers only spurious errors.

                                              The additional cultural context that’s helpful here is that BEAM systems are (much to the annoyance of devops folks trained in the way of the times) handled as somewhat living critters. It is not uncommon (hell, did this last month during peak traffic with my team) to shell into a running system, handling production traffic, and debug and tweak and patch running code. There is a different mindset about this sort of things than I believe we have when working with bake-and-deploy-and-scrap languages and runtimes.

                                              I agree with your interpretation of “let it crash”, but having also suffered through a lot of oddly-engineered Elixir (which is similar enough to Erlang in the relevant ways I feel comfortable saying so) that has forgotten or misunderstood “let it crash” I do have sympathy for and empathy with the author.

                                              1. 5

                                                +1

                                                This article, and these comments, seem to be conflating multiple notions of crashing. I guess crashing is usually understood as something that happens at or above the level of an OS process.. That is, a function call that doesn’t return successfully isn’t usually described as having crashed. Or, an autonomous task within a process that hits a terminal error isn’t typically said to have crashed. A process crashes. A system crashes.

                                                Erlang defines a system model in which a supervisor (BEAM/OTP) manages a constellation of redundant entities (actors) which each serve isolated workloads (typically requests). When Erlang says “crash” it means something quite different, and far less impactful, than when Rust or Linux says “crash”. And Erlang’s definition is niche.

                                                Maybe this is a distinction that everyone here fully understands and I’m harping on a pointless point, could be. But it’s definitely misunderstood as a general concept, and IMO shouldn’t be advocated-for without serious qualifications.

                                              1. 4

                                                I always found the “Let it crash” very convincing. One of the most convincing talks I know is not even about Erlang. But I’ve never actually coded anything substantial in either Erlang or Elixir myself. I went the other way instead with Rust and static typing.

                                                But I’m curious, for those of you who do work on such systems, does it deliver on its promise? Is it simpler? More robust? And is modern Elixir done in the same vein of “Let it crash” and very few tests or verification?

                                                1. 5

                                                  Rust follows the “let it crash”-philosophy, its panic system is Erlang-inspired. I used to be even stronger baked into the language, when it still had language-level tasking with a runtime. The nomicon chapter on unwinding still calls it out

                                                  You can see that in the tasking/threading APIs where a panic crashes that component and another part of the system is responsible for handling.

                                                  1. 4

                                                    I’ve had to deal with more than one Rust service that take this philosophy to heart and so will fully crash the entire program in the presence of, say, a network connection timeout to a non-business-critical API endpoint. Maybe this isn’t the intended effect of the panic approach to error management, but it does seem to be a common outcome in my experience.

                                                    The problem here is a mismatch of expectations. It’s nominally OK to crash an Erlang actor in response to many/most runtime faults, because Erlang actors always operate in a constellation of redundant peers, and failure is a first order concern of their supervisor. That crash impacts a single request.

                                                    But e.g. systemd is not the OTP, and OS processes don’t operate in a cluster. A service running as an OS process is expected to be resilient to basically all runtime errors, even if those errors mean the service can’t fulfill its user-facing requirements. If an OS process crashes, it doesn’t impact a single request, it impacts every request served by the process, every other process with active connections to that process for any other reason, assumptions made by systemd about the soundness of that binary, probably some assumptions about off-host e.g. load balancers shuttling traffic to that instance, everything downstream from them, etc. etc.

                                                    If “crash-only” means “terminate the request” and not “terminate the process” then all good! But then “crash” isn’t the right verb, I don’t think, as crashing is pretty widely understood to mean the OS level process of the program. Alas.

                                                    1. 7

                                                      Yeah, I think this is an acute case of catchy, but wildly misleading terminology. What is really (as in Elang or Midori) understood as proper “let it crash” is two dual properties:

                                                      • abandoning the current “thing”
                                                      • containing abandonment to some well-defined boundary, such that:
                                                        • abandonment don’t propagate outside of this boundary
                                                        • tearing things down at this boundary doesn’t compromise the state
                                                        • restarting at the boundary is a well-defined operation which can fix transient errors
                                                        • the actual blast radius from abandonment is small

                                                      Everyone gets the first point, buts it’s the second one which matters, which is hard, and which leads to simplicity and reliability.

                                                      1. 3

                                                        To expand on this, Rust does only marginally, if at all, better here than you average $LANG:

                                                        • the build-in boundary is OS thread, which is often too coarse-grained, there’s catch_unwind for do-it-yourself boundaries. There’s nothing to protect from thread monopolizing the CPU due to an infinite loop bug. Some errors (stack overflow, OOM) abort the process bypassing the recovery mechanism.
                                                        • UnwindSafe machinery in theory helps somewhat with the tainted state problem. In practice, it’s too cumbersome to use and people often silence it. I had one spectacular bug where UnwindSafe would’ve saved couple of days of debugging, if it wasn’t silenced due to it tripping a compiler bug.
                                                        • nothing to make restart workable, do-it-yourself again.
                                                      2. 2

                                                        But e.g. systemd is not the OTP, and OS processes don’t operate in a cluster. A service running as an OS process is expected to be resilient to basically all runtime errors, even if those errors mean the service can’t fulfill its user-facing requirements.

                                                        I think this might be oversimplifying. Whether it’s reasonable to let a service continue to fulfill tasks despite encountering a serious fault is not clear cut. Example: A service that has a lot of shared state, say thread bound caches of various sensitive user data, a crash might lead to failed cleanups and subsequent data leaks.

                                                        1. 1

                                                          Let me rephrase my claim to be more precise.

                                                          A service running as an OS process is generally expected to be resilient to runtime errors. If a runtime error puts the service in a state where it can no longer fulfill user requirements, and that state is transient and/or recoverable, it is usually preferable for the service to continue to respond to requests with errors, rather than crashing.

                                                    2. 4

                                                      In my experience across several companies and codebases in Elixir, I’d say the following things.

                                                      “let it crash” can lead to clean code. It also originated out of a design space that I believe doesn’t map as directly onto modern webshit as folks want to believe. This is neither good nor bad, it’s just an occasional impedance mismatch between system design philosophies.

                                                      “let it crash” encourages some people, drunk on the power of OTP and the actor model, to grossly overcomplicate their code. They decide deep supervision trees and worker pools and things are needed when a simple function will do. This is the curse of the beginner Elixir or Erlang developer, and if properly mentored this goes away quickly. If not properly mentored it progresses to a case of conference talks and awkward libraries.

                                                      Testing and verification in the BEAM ecosystem is weird, and until recently was both best and worst in class depending on what languages you were up against. Dialyzer for example is a marvelous typechecker, but there is growing suspicion that it is severely stunted in the sorts of verification it is categorically capable of. On the other side, property-based testing is strictly old-hat over in at least the Erlang ecosystem and has been for quite some time iirc. Other folks are catching up.

                                                      (Testing is also–in my opinion–most often valuable to solve team coordination problems and guard against entropy caused by other humans. This is orthogonal to language concerns, but comes out when you have larger webshit-style teams using BEAM stuff when compared with the origin of Erlang.)

                                                      Robustness is quite possible. I’ve alluded elsewhere to how a running BEAM instance is more of a living thing (gasp, a pet!) than most contemporary app models (pour one out for Smalltalk)…this unlocks some very flexible things you can do in production that I haven’t really seen anywhere else and which make it possible to do things without downtime during an incident that most folks would just look at and go “wat.”. On the other hand, you have to design your systems to actually enable robustness–writing your standard webshit without structuring the application logic to have affordances for interactivity or process isolation or whatever means you’re basically using the BEAM like you would other more conventional systems.

                                                      (You can also, as I’ve done with Python on at least one occasion, build a BEAM-like actor model with fault tolerance. The idioms are baked into Erlang and Elixir, but you can with sufficient effort reproduce them elsewhere.)

                                                      (Also, “let it crash” doesn’t mean you won’t sometimes have to wrap your whole VM in a systemd unit to restart things when, say, an intern pushes to prod and blows all the way up the supervision tree.)

                                                      1. 2

                                                        In a sense, yes—an example I’ve run into several times is that a service you depend on becomes intermittently unresponsive. In a “regular” software service, unless you clutter your code up with retry and “fail soft” logic (basically your own ad-hoc OTP) this usually means a hard error, eg a end-user received an error message or a service needed to be restarted by the OS process manager.

                                                        In Erlang the system can usually deal with these kinds of errors by automatically retrying; if the operation keeps failing, the error will propagate up to the next level of the “supervision tree”. Unless it makes it all the way up to the root the application will keep running; sometimes the only indication that something went wrong is some log output.

                                                        The nice thing about “Let it crash” is that you don’t have to consider every possible failure scenario (eg what happens if this service call returns malformed data? What if it times out?). Instead of trying to preempt every possible error, which is messy and basically intractable, you can focus on the happy path and tell OTP what to do in case of a crash.

                                                        That said, “Let it crash” is not a silver bullet that will solve all errors in your distributed system; you still have to be acutely aware about which parts of the system can be safely restarted and how. The nice thing is that the base assumption of OTP is that the system will fail at some point, and it gives you a very powerful set of tools to deal with it.

                                                        Another thing that makes Erlang more robust is the process scheduling model: Processes are lightweight and use preemptive multitasking with fair scheduling, which means you’re less susceptible to “brownouts” from runaway processes.

                                                        1. 1

                                                          But I’m curious, for those of you who do work on such systems, does it deliver on its promise? Is it simpler? More robust? And is modern Elixir done in the same vein of “Let it crash” and very few tests or verification?

                                                          I can only speak to the first half, and it is completely dependent on the team culture. If the team is very OTP/Erlang native, it can work out incredibly well. These teams tend to be extremely pragmatic and focused on boring, obvious, naive ways to solve problems, using global state as needed even!

                                                          However, when OTP/Erlang collide with a team trying to treat it like anything else things can go badly…. quickly.

                                                        1. 6

                                                          This is all too exciting technologically, but a point that is rarely discussed is: the web is not centralized out of (pure) evilness from corporate giants.

                                                          The single key reason for centralization, I believe, lies in economics: larger players enjoy gains from scale, are further down the learning curve and in some cases can form network effects (positive externalities).

                                                          It is simply cheaper to run larger data centers, cumulative experience matters a lot in software development, and network effects are evident in market places and social media.

                                                          The risks from concentrating too much power in the hands of a few giant corporations must be balanced against the costs of running smaller tech organizations.

                                                          Or course novel technology can reshape the economics that lead to centralization, but I think most discussions miss the economic causes.

                                                          1. 9

                                                            The network itself remains very important.

                                                            I remember the advent of ADSL in France. Emphasis on Asymmetric DSL. Before that everyone was on Dial up, and cell phones were still fairly uncommon. And since Dial-up modems basically hang up your phone while you’re connected, no one left their connection open indefinitely. Having a server at home was thus a pretty big no-no. The market picked up on that, and concluded people didn’t “want” to have servers at home.

                                                            So when DSL came about this all “wise” and “efficient” market sacrificed upload bandwidth to give customers more download bandwidth. Dial-up showed they don’t “want” to run servers anyway, and the fact DSL connections can let us use our regular phone at the same time totally won’t change anything. Or so the market thought. And what do you know, the market was right: people still didn’t run servers at home, and it totally had nothing to do with the asymmetry of the bandwidth… you get the idea.

                                                            Starting from there, there was no market for servers, low-power servers, easy to use servers, distributed social networks in a box, photo and video sharing from a home server… you get the idea: had bandwidth been symmetric from the start, peer-to-peer protocols like BitTorrent would have entirely voided the need for all centralised services except search. And it’s not at all certain that the global costs would have been any higher: with centralisation, data tend to travel longer distances, and that’s not free. Besides, pretty much everyone has a router at home, that is beefy enough to host all kind of services. And with peer-to-peer protocols it wouldn’t even need to scale.

                                                            But with the network we have now, I guess I’ll have to let Google and Amazon decide what I want to eat for breakfast…

                                                            1. 3

                                                              I’m not defending market allocation as wise and efficient, I’m pointing to benefits of scale, learning and network effects. There are lots of suboptimal equilibria in economics.

                                                              1. 3

                                                                “suboptimal equilibria” is an interesting way to refer to the horrors of the last century of globalized markets

                                                                1. 1

                                                                  It’s a technical term in economics, but there’s a cool example that shows it is not related only to markets.

                                                                  If you have two ISPs, each covering part of a graph of nodes, each has the incentive to deliver a packet as quickly as possible to the other ISP. Then the packet goes through the other ISP possibly making the combined path large than the shortest path. Both ISPs are worse off if there is no regulation.

                                                                  I feel the sarcasm in your comment, but I think it as detrimental to the economist profession as abusing open source contributors is detrimental to the software development profession. For that reason, I would like you to the consider the following arguments, because they might convince you.

                                                                  (For the record, my general political stance is to the left, and I loathe the egoistical morals that arise strongly in a free market economy.)

                                                                  1. that the “horrors of globalized markets” must be compared to alternatives, i.e., non market economies, for which only bad examples exist nowadays. Soviet economies in the long run collapsed because central planning is really too complex. China can be called as an example but it does feature markets. Primitive, pre agricultural societies are speculated to have featured a better quality of life than today, but hunting and gathering do not scale. That is, the horrors must be compared to the alternative horrors.

                                                                  2. that the “horrors of the globalized markets” are mostly attributable to fraud, greed, lack of proper regulations and governance, and not attributable to markets as a mechanism for allocating resources.

                                                                  3. that new investments that are more efficient can instantly depreciate older, less efficient investments in capital that is not reallocatable. If you invest in a new factory that is much more efficient, old factories might become worthless before old factories are paid back. If you hire cheaper workers, more expensive local workers might not be able to find a new occupation; that is one of the negative outcomes of competition, and still characterizable as suboptimal equilibria.

                                                                  1. 1

                                                                    How can any particular act of fraud or greed be described as exterior to that act’s historical social conditions? You argued that “alternatives” to markets are necessarily worse off because of their worst anecdotal/particular features; if there is No True Scotsman for the social conditions associated with markets, why is the same not true in your view for the social conditions associated with non-markets? It also gives me the question: do you or do you not believe that the behavior of people is the product of their environment?

                                                                    Honestly, I think here not about just “the market” or really even “globalized markets” but seriously speaking I think of the history of capital accumulation instead because it is, in my opinion, not only easier to define and identify discretely than some idealized social system of “capitalism” or an abstract notion of trans-historical market relationships (“a mechanism for allocating resources”) but can be easily understood as particular historical events whose effects, such as materially unnecessary suffering, can be directly observed. I look across that definite history, and I see a violently self-expanding phenomenon, whose blood trails aren’t at all accidental or difficult to correlate. I can come up with a litany of my own views on the particulars, which I definitely prefer to vague conversations about abstract signifiers, but I have yet to be satisfied by any argument to the contrary. No person has (or likely ever could) convince me, for example, the Bhopal Disaster was just accidental to the process of capital accumulation. You may not even disagree with me when I’ve worded my view on this in this way, but that is at least how I see it.

                                                                    Additionally, I don’t understand how the cheapening of labor can be seen as a “suboptimal equilibria” of any real market when it appears in measuring our ongoing epoch of industrial capital to be a tendency of all mass forms of waged labor (which become increasingly dominant), with the temporary and localized exception of class struggle, such as through unionization or the passage of beneficial legislation, stalling the ever-falling value of living labor in the face of a ceaselessly growing mass of dead labor.

                                                                    1. 1

                                                                      that the “horrors of globalized markets” must be compared to alternatives, i.e., non market economies, for which only bad examples exist nowadays.

                                                                      If we stick to totalitarian examples only. Partially non-market economies aren’t that awful. I’m speaking specifically of the public sector amidst an otherwise market economy: administration, police and justice (that you’d find even in a “minimal state”), but also firemen, schools, roads, trains, energy, water, health care, unemployment insurance, retirement plans. I’m sure I’m forgetting some.

                                                                      Point is, most USA citizen would recoil in horror at the “Socialism” implied by taking all of the above out of the market, and give it to this inevitably bad state “monopoly”. But it worked in France for decades. Many of our problems come from dismantling this public infrastructure and giving it to the market.

                                                                      In fact, whether something should be given to the free market or commanded by the state (or local public institutions) should be decided on a case by case basis. For instance, internet providing should probably be a mix of the two: the cables themselves are a big investment, best undertaken by the state (and it should be optic fibre with symmetric bandwidth, since (i) symmetric is Good™ and (ii) symmetric fibre is the cheapest option anyway). To pay for it the state operator leases the bandwidth in those cables to anyone who would buy those slices. With one very important caveat: flat rates. That setup has been experimented in some regions of France, and the result was a plethora of internet providers, both commercial and non-profit. That’s how you make a free and uncontrollable internet.

                                                                      1. 1

                                                                        Yes, completely agree: public goods should be provided by the state, including the Internet. The economic analysis is that of positive externalities. “minimal state” is not good even in theory, because it excludes the possibility of providing goods that are more efficiently provided by a single, internalizing actor.

                                                                        What’s your view on universal basic income?

                                                                        I tend to favor it.

                                                                        1. 1

                                                                          There are 2 competing views for unconditional income. Basic Income stems from the idea that we are beings of need, and need some unconditional minimum income even if we don’t contribute. It’s mostly a way to lessen the horrors of capitalism without actually ending it. The other view comes from some form of communism or classical anarchism: it’s called qualification based salary (a mouthful), and it stems from the idea that we are productive being, and every citizen ought to receive a salary for this productivity to be recognised. It’s part of an alternative to capitalism.

                                                                          I’m personally not sure where to lean exactly. But there’s one thing we must recognise as soon as possible: our current way of life is not sustainable, and if we keep this up we’re facing not just the collapse of our civilization, but possibly even the extinction of humanity before the end of the 22nd century. Whatever we do, we must not cross the hard limits that will prevent the Earth from being inhabitable by humans, and if at all possible, we should at once preserve what can be, and sacrifice what threatens it.

                                                                          The primary cause of that pending doom, namely the way we produce and consume goods, also known as our economy, has a name: Capitalism. It cannot be saved in the long term, and must be sacrificed. Merely trying to reining it in will either not work, or bind it so tightly it might not be called “capitalism” any more. And then we must pursue a life that doesn’t consume as many resources, yet live long and happy lives, and keep hope for a brighter future.

                                                                          I’m not certain how to go about it, but the usual defeatism about capitalism being the only alternative does not help. As is branding any proposal that’s not capitalism as “we’ve tried this before, it was worse”. Thing is, keeping up with capitalism is likely to reduce the world’s population to a tenth of its highest peak in less than 2 centuries. In practical terms this means war, famine, and illness. Mostly the last 2, fuelled by the first. So now we get to pick our poison: change the system and risk some new variation of some horrible totalitarian rule, or not change the system and risk the Malthusian consequences.


                                                                          If you’re still with me so far, and can entertain the idea that we ought to find alternatives to capitalism, we can start thinking how to proceed from there: what do we want? And how do we decide it?

                                                                          One such proposal, that we can call “communism” though it’s very different from what we have seen in USSR and China, start with that qualification based salary. The idea is giving everyone their minimum income at their political majority. Then, as their “qualification”, we would increase that income up to some maximum. Typical proposals for that maximum range from 3 to 6 times the base salary. Note that in France, less than 10% of the people are paid more than 3 times the minimum wage, and less than 5% are paid more than 6 times that. So a limit of even 3 isn’t that unreasonable.

                                                                          One reason for that qualification, as opposed to an equal allocation for everyone, is that there is work to be done, and some of that work is a combination of painful, boring, dangerous, unrewarding, specialised… one way or another it’s stuff that not everyone will want to do. Raising one’s qualification is an incentive to do that work. Who exactly decides how and when to raise someone’s qualification is an open problem. Some people will inevitably try to game or cheat the system, so as in everything politics we’ll need some kind of trustworthy process, checks and balances and all.

                                                                          Then we need to decide what actually needs to be done. Food and shelter, education, infrastructure, whatever we need to live well enough while making sure we do not render our planet uninhabitable. That probably means ditching a few things, such as airliners or short-lasting electronics (no more buying a new iPhone every couple years). And as much as I love my own VR setup, we may have to stop producing high-performance computers for mere entertainment purposes (as amazing computers are, they’re one of our most polluting industries).

                                                                          Then we need to decide how it needs to be done. And even though so far I’ve been talking “communism”, we did try command economies, and that didn’t go so well. Instead the workers should probably decide how they work. We would abolish lucrative property (getting dividends just because you happen to “own” the factory people are working in), but keep usage property (those who work in the factory decide how the factory is actually run). This means localizing and distributing what can be. A bakery for instance has a very limited radius of influence. In a densely populated area it would be barely half a klick. The stakeholders of that bakery would then be all the people within its sphere of influence. They could decide the price of bread, and how much the bakery should be subsidised, if at all. And if it turns out the bakery is profitable, great, those profits goes back to the community to subsidise something else (possibly including an increase in the bakers’ qualifications to reward their good work).

                                                                          A nuclear plant on the other hand should be managed at the grid level, and that grid (at least in the EU) has international reach. That’s quite a different game. Anyway, this system is bound to be very complex with lots of disputes, and I bet different areas or cultures would make very different choices. But by keeping things local we change one crucial thing: people can meaningfully participate in politics again (the price of bread after all is a meaningful political decision). If we are to get out of the political apathy I see around me (including, let’s not lie, myself), this ability to actually participate is a necessity.


                                                                          What I think of Universal Basic Income? I’d wager it’s a necessary component of a much, much larger whole.

                                                                  2. 3

                                                                    You don’t get my point: one huge reason for the benefits of scale to realise in the first place, is the asymmetric nature of the network, and the centralised nature of the web (one web server for many web clients). If we had symmetric bandwidth from the beginning of cable & DSL, coupled with a web that used a peer-to-peer transport network, the story of the internet would have been very different.

                                                                    About the other two… yeah, network effects are a big one, but they become less relevant in federated networks such as email (except when the big ones start to discriminate small servers), and not a problem at all in truly distributed networks such as BitTorrent. I’m sceptical about knowledge transfer within a single company. I’ve been programming for money for 15 years now, and no company I have ever worked with transferred any significant amount of knowledge. Trivia specific to the company or the project, sure, but nothing fundamental beyond what I’ve learned at school or on my own.

                                                                    1. 3

                                                                      What I think is even more primary than assymmetry of bandwidth, is assymmety of connections. Assymetry of bandwidth IMO just grew from the fact that regular consumers were not allocated a scarce resource that a public static IPv4 address is. In order to address that scarcity, solutions like NATs started to develop and took the web in the direction where is it is not true that if you can connect A to B, then B can also connect to A. Assymmetry of bandwidth seems to me to be a consequence of that. I totally agree though that the internet would look totally different had we lived in a symmetric world. IPv6 offers such a perspective.

                                                                      1. 3

                                                                        Good point, I got tunnel vision here. Offering a single IP to consumer then the subsequent need for NAT is indeed worse than the bandwidth story. I think it stems from similar causes (nobody has a server at home during dial up times), but I agree the consequences are even worse.

                                                                        Now I almost hope IPv4 becomes unusable enough that everyone switches to IPv6. But I’m afraid we’ll just generalise Carrier Grade NAT instead. In my opinion the regulators should step in: “Internet providers must give each customer at least one public, static /64 IPv6 range, with no restriction”.

                                                                      2. 2

                                                                        Bluntly, the availability requirements of content consumers, in general, cannot be satisfied by people hosting their own content. Hosting requires specialized knowledge and consistent attention that simply can’t be provided by individuals.

                                                                        Centralization delegates these complex requirements to a third-party, which can effectively satisfy them, at an organizational level, without burdening the content producer with the details. It’s an abstraction, basically, which models the perspective of its users. Even if all of the technical costs of self-hosting were reduced to zero, people would still choose centralized hosting.

                                                                        1. 2

                                                                          Bluntly, the availability requirements of content consumers, in general, cannot be satisfied by people hosting their own content.

                                                                          What availability requirements exactly? Sure I like YouTube to be available all the time, but I’m certainly willing to tolerate some of my favourite channels going down from time to time. Which is what will happen with self hosting, and that’s fine.

                                                                          Hosting requires specialized knowledge and consistent attention that simply can’t be provided by individuals.

                                                                          Hosting with 99.99% availability, sure. In practice, 99.9% is more than enough, and easily achieved with an auto-updating box, even with the odd power outage, or even hardware failure if you can replace the parts fast enough.

                                                                          Of course, countries with less reliable power will have more problems.

                                                                          Even if all of the technical costs of self-hosting were reduced to zero, people would still choose centralized hosting.

                                                                          I blame political apathy.

                                                                          1. 1

                                                                            Self-hosting on private uplinks delivers p90 zero-nines availability. The set of people capable of operating a one- or two-nines appliance in their home is statistically zero.

                                                                            1. 2

                                                                              Let’s count, shall we?

                                                                              • One nine means 10% unavailability, or less than 37 days per year.
                                                                              • Two nines means 1% unavailability, or less than 4 days per year.
                                                                              • Three nines means 0.1% unavailability, or less than 9 hours per year.

                                                                              I have a router at home (the one that provides internet access, and in about 3 years it only went down for a couple days because the connection was cut off. That’s well over 2 nines, with zero effort on my part. My grandma who knows nothing about technology, gets similar performance.

                                                                              I also have a NAS that I share with my brother (installed in my home, accessed by both, and so far the only reason for it to go down was when I moved it from one room to another (5 minutes, so this still allows for 5 nines), or when my connection itself went down. Also zero effort beyond the initial setup.

                                                                              Going beyond that will require some distributed backup tech. Something basic like doubling disk usage and sharing with a single trusted friend or family member would solve most problem and add 1 or 2 nines right off the bat. And I bet something fancier like Reed-Solomon codes could deliver similar reliability while sacrificing much less. Of course it needs to be developed, but the maintenance effort? There isn’t much of one.


                                                                              You keep asserting that people need skill to host stuff at home. They don’t. They just need the right appliance. The ease of use of centralised system can extend to “host-at-home” systems as well. If you disagree, please name specific skills people would need, and why those couldn’t be addressed by NAS/router vendors.

                                                                              And if I may, the centralised systems aren’t that reliable either: they often take down your content, or even end your account, for reasons that aren’t always under your control. And some content deemed inappropriate to the biggest platforms can’t be hosted there at all. Not for technical reasons of course, but we ain’t ever going to find Bible Black on YouTube even if the authors wanted to.

                                                                          2. 1

                                                                            Most torrents I’ve downloaded are an order of magnitude more available than anything ever hosted in CDN’s honestly. You could argue that the complexity is offloaded to the tracker but honestly running something like a tracker is dead simple these days with docker and such.

                                                                            1. 1

                                                                              Most torrents I’ve downloaded are an order of magnitude more available than anything ever hosted in CDN’s honestly. You could argue that the complexity is offloaded to the tracker but honestly running something like a tracker is dead simple these days with docker and such.

                                                                              To be clear, you’re saying the user-experienced availability of data from a .torrent file — typically magnet links or whatever? — is an order of magnitude higher than the availability of content on CDNs like Akamai, Cloudflare, or Fastly?

                                                                              1. 2

                                                                                Duh, because the files in the CDN are often taken down by their owners for financial or other reasons. One CDN provider’s network goes down (happens) the files go down as well, while torrents require every single peer or tracker on a file to fail.

                                                                                1. 2

                                                                                  the files in the CDN are often taken down by their owners for financial or other reasons

                                                                                  There are two dimensions availability under discussion here.

                                                                                  1. Availability of content which is currently authorized by the owner for consumption
                                                                                  2. Availability of content which was once, but is not currently, authorized by the owner for consumption

                                                                                  First, let’s scope these categories. How much content do you believe exists in each of them? (This reduces to: how much content do you believe is generally requested, which has been taken down by the producer?) 50/50? 90/10? 99/1?

                                                                                  (Secondary question on that point: do content producers not have the right to delete content they once made available? If not, what are the consequences on the sovereignty of data?)

                                                                                  Then, let’s define availability. If I request a file at time t0, then availability means it should arrive in full by some time t0+N, else it is unavailable. What’s a reasonable N? “Indefinite” is not a reasonable answer.

                                                                                  I claim that a reasonable answer to the ratio-of-content question is at least 99/1, and more likely 99.999/0.001; and a reasonable answer to the availability question is at least O(single-digit seconds), and more likely O(sub-second). Through that framing, I don’t see how torrents would ever provide a better user experience.

                                                                                  I’m sure you don’t agree with this framing. But I think that just speaks to the use cases we’re each starting from. The current internet provides roughly 100ms response latency to individual requests. No rational actor will opt in to a system with inferior performance, absent extraordinary circumstance. So it seems to me that no system that doesn’t meet or beat that performance benchmark can be considered viable.

                                                                                  1. 2

                                                                                    I know the use cases were different. I just personally don’t care about the former (business-oriented) case while understanding it is important for other people who are not me. Agree on the latency problem for smaller files though. Every decentralized system out there is non-ideal in one way or another (and I don’t mean generally as I don’t believe there is a such thing as a “general” use-case). I also think the “centralized”/“decentralized” dichotomy is quite telling in itself due to the natural tendency of capital investments to centralize. Computer systems only care about such a thing insofar as they are created as a part of an economy; there’s nothing technically speaking that says data or computation must be perfectly distributed or perfectly centralized. Distribution is just a tool that’s deeply underutilized due to the practical dominance of market interests in our society.

                                                                                    1. 2

                                                                                      If we’re talking about systems that are meant to be used by a non-trivial number of human users, then the concerns which become very important, and which generally disqualify decentralized systems, are addressability and availability.

                                                                                      Addressability means that users should be able to efficiently interact with the system via identifier(s) that are humane and authoritative. Humane means something like “can communicate it in a spoken conversation” which is satisfied by a URL or phone number, and not by an IP address or content hash or Bitcoin wallet address. Authoritative means that my youtube.com better be the same as your youtube.com (net, at least).

                                                                                      DNS is authoritative. Tor, Onion, ENS, etc. are not.

                                                                                      Availability means that users can rely on the service to work, to their level of expectation. People expect that a physical bank might be closed from 5pm to 8am, but anything on the internet is expected to work 100% of the time. There’s some tolerance for outage but it’s maybe like 1%, anything more than that and the service is understood as unreliable.

                                                                                      A website with a team of on-call engineers distributed across timezones is available. A video hosted on a home internet connection is not.

                                                                                      I’m not aware of any way for decentralized systems to satisfy these requirements. Certainly no decentralized system existing today does so.

                                                                    2. 3

                                                                      It worth separating centralisation in terms of infrastructure and in terms of governance and administration when considering economics. Cloud providers pass on a lot of the cost savings from economies of scale and, as long as there is more then one, they will keep being pressured to do so by the market. With modern FaaS and disaggregated storage offerings, you can build things that are a lot cheaper to run than a single VM (let alone a real computer connected to the Internet). You may be limited to 4-5 cloud providers that can host it cheaply, but thousands of users can deploy separately administered instances of whatever other thing is.

                                                                      1. 2

                                                                        Enormous agreement re: broad misunderstanding of the forces that motivate centralization. And I also totally agree that the economic perspective is underemphasized. But I think, if we’re talking about systems with a non-trivial number of (human) participants, then centralization is both inevitable and necessary, for sociological reasons.

                                                                        I may be able to negotiate consensus with my neighbors to decide who will host the yearly block party, but that stops working beyond an approximately Dunbar-number-ish group of participants. If a system needs to accommodate more than that number of users, then it must define abstractions, and operations on those abstractions, to be practical. You can’t get those things without delegation of trust to some kind of authority.

                                                                        Concretely: if I want to propose a vegan option for the block party, I can go to 10 neighbor’s houses and make my case; if I want to propose closing the border to Canada, I cannot realistically canvas every citizen in America to make my case. Equivalently, if I want to make some change on Ethereum, I can’t realistically persuade every participant in the open network, I have to go through some governance mechanism, which is necessarily well-known, well-defined, respected by most/all participants — i.e. centralized.

                                                                        IMO no matter how you shape the economics, or iterate on the technology — no amount of BFT protocol cleverness, or trust-less encryption shenanigans, or whatever — it’s fundamentally not possible to decentralize systems at scale.

                                                                        1. 5

                                                                          You’re talking about governance here. When the limitations are more technical, decentralisation is very possible. For instance, there would be no need for YouTube if:

                                                                          • Our bandwidth was symmetric from the start.
                                                                          • All videos were distributed through BitTorrent or similar.

                                                                          Instead everyone would be able to host their videos from their home connection at relatively little cost. A centralised video search service would probably still have emerged, but the only reason hosting itself is centralised is because the web is centralised.

                                                                          1. 3

                                                                            I don’t buy this argument. It assumes technically capable users who are able and willing to have their machine on at all times. Even most techies don’t want the hassle of having a server in their cupboard (which also requires maintenance to be kept secure and also hardware will fail).

                                                                            You could argue that specifically bittorrent would be able to deal with nodes dropping out and coming back up, but that assumes there are enough people willing to sacrifice disc space and upload bandwidth with Joe Schmoe’s home videos to ensure there’s always someone’s machine online when you want to watch such a video. Don’t underestimate the convenience that hosted solutions offer.

                                                                            This is also the same reason more and more goes into “the cloud” even though that’s not technically necessary. There were actually probably more home servers in the late nineties/early zeroes than nowadays.

                                                                            1. 5

                                                                              It assumes technically capable users who are able and willing to have their machine on at all times.

                                                                              Yeah, there’s just one snag: it’s already the case. Right now, my grandma has a machine at home that is always on, and always connected to the internet: her router. Now I’m not asking that people actually administrate their home servers. There is such a thing as usable software. The only reason having a server is such a PITA right now is because of that very router, with NAT, firewalls etc… whose configuration interface isn’t under the control of any software provider (another avoidable reason for decentralisation: even if I play a 2-player game the only usable option is hole punching the NAT with the help of a central server… or even use a central server from the start).

                                                                              You could argue that specifically bittorrent would be able to deal with nodes dropping out and coming back up, but that assumes there are enough people willing to sacrifice disc space and upload bandwidth

                                                                              The upload bandwidth is only a problem because our bandwidth is asymmetric. If it was symmetric, people would naturally upload just as much as they download, supply would naturally equal demand, and it would all scale up and down without any problem. The disk space would not be a problem either: if everyone just uploaded what they were downloading right now (thanks to symmetric bandwidth), they would only “sacrifice” space to store stuff they actually want. And they can stop sharing not long after having downloaded what they need.

                                                                              As long as there’s one primary source in the world that is dedicated to distribute something they think is important (such as the videos of my cute cat), anyone who want to download it, can. That part is not much different from the web, actually. It just scales better without imposing all the costs on the distributor.

                                                                              Alas, our bandwidth is asymmetric and people have little upload to spare. In this world (our world), you’re correct. My conclusion from this is that asymmetric bandwidth is one of the biggest causes of internet becoming ever more centralised. That, and the hegemony of HTTP.

                                                                              1. 2

                                                                                And the router my parents have at home occasionally (once a month?) craps itself and has to be restarted. I know because my off-site backups stop working and I have to ask my dad to go and do stuff with it. We have symmetric bandwidth, or at least high enough upload, in some places and it doesn’t change things: very few people host stuff at home.

                                                                                1. 2

                                                                                  And the router my parents have at home occasionally (once a month?) craps itself and has to be restarted.

                                                                                  Sure. Nothing is perfectly reliable. But thanks to asymmetric bandwidth, there never was a market for 99.99% availability in home routers/servers.

                                                                                  We have symmetric bandwidth, or at least high enough upload, in some places and it doesn’t change things: very few people host stuff at home.

                                                                                  Again, no market. And even if we solve the asymmetry now, the cloud players are too entrenched by now.

                                                                            2. 2

                                                                              there would be no need for YouTube if our bandwidth was symmetric from the start [and] all videos were distributed through BitTorrent or similar

                                                                              The assertion here is that if we can reduce bandwidth costs to zero, and provide a practical solution to addressability, that most people would choose to self-host rather than uploading content to centralized services?

                                                                              1. 2

                                                                                I’m not making any claim about now. I’m making a counterfactual claim about the past: if bandwidth had been symmetric, and we had usable server boxes from the start, most of those centralised services would likely not have risen to power to begin with. (A single box provider probably would have though, just like Microsoft did with Windows. But I think it would have been a lesser problem.)

                                                                                Now that they have however, switching away from them is going to be a totally different game. Without some strong regulation (say, mandate adversarial interoperability for stuff like Facebook), I’m not sure this is even possible. They’re too entrenched now.

                                                                                1. 1

                                                                                  if bandwidth had been symmetric, and we had usable server boxes from the start, most of those centralised services would likely not have risen to power to begin with.

                                                                                  Keeping a server up over time requires continuous attention and specialized knowledge which is unavailable to everyone except a tiny niche of people. These costs dominate the calculus that drives hosting decisions. No amount of UX improvement or bandwidth cost reduction will allow a .mp4 hosted on my mother’s home internet connection to meet user expectations. Two nines, hell, one nine, is uptime that’s only possible with round-the-clock on-call engineering staff. And that’s only possible to provide in centralized organizations. Centralization isn’t some pathological side-effect of market forces, it’s Actually Good™ because it delivers net better results to all stakeholders.

                                                                                  1. 2

                                                                                    Keeping a server up over time requires continuous attention and specialized knowledge which is unavailable to everyone except a tiny niche of people.

                                                                                    I’m repeating myself, but you really need to state the specialized knowledge you’re speaking of.

                                                                                    I’ve had a router at home for over 15 years now, and the only reason it ever went down was because I was moving to another house, or because the line itself went down. That’s between 2 and 3 nines out of the box, before we even speak distributed backup. (2 nines means 1% downtime, or 3 days and 15 hours per year). Nobody was ever on call for that. And both my Mom and my Grandma enjoyed similar availability on their own connections (likely a bit more since they haven’t moved out in the last 20 years).

                                                                                    I’m also not sure why you’re talking about bandwidth costs at all: sure to deliver stuff fast you need enough bandwidth, but protocols like BitTorrent aren’t limited by the origin’s upload, it’s limited by everyone’s upload. If everyone can download at 1MB/s and upload at 100KB/s, the average torrent will download at 100KB/s. But if both upload and download are capped at 500KB/s, then download will reach 500KB/s. And the beautiful part is that it doesn’t matter whether there’s 1 peer or 1 million: everyone gives a bit of their upload, so increasing a server’s popularity does not increase their bandwidth costs.

                                                                                    But we got asymmetric bandwidth, so the network as a whole had much less upload capability, and so in our world torrents tend to download fairly slowly. Much slower than something hosted centrally. But that’s only because our network is asymmetric, and therefore optimised for centralisation.

                                                                                    Centralisation isn’t an inevitability inscribed in the laws of physics. If anything in fact, I would guess the laws of physics (specifically the speed of light) would likely favour decentralisation, because it allows data to go through shorter paths without necessarily needing CDNs or similar huge cache systems.

                                                                                    1. 2

                                                                                      Centralisation isn’t an inevitability inscribed in the laws of physics

                                                                                      Physics, no. Systems used by humans, yes, it actually is.

                                                                                      1. 2

                                                                                        How then? Is is something as vague as “people seek power at the expense of others, therefore inequality”? Or do you have actual mechanisms in mind?

                                                                                        1. 2

                                                                                          It’s not a question of power or equality, it’s a question of human nature.

                                                                                          Humans interacting with systems express intent, and evaluate outcomes, invariant of the implementation of those systems. When I send a payment to Visa, I have the domain concepts of “me” and “Visa” and “my payment”. If I fat-fingered the recipient of my payment, it’s absolutely necessary that I can petition a well-defined, singular, authoritative Visa entity to fix that mistake.

                                                                                          You can’t assert that Visa is a decentralized abstraction, and so has no singular — centralized — authority that I can address when there is a problem, and thus that my mistake is just too bad and I have to deal with it. That system is inhumane and non-viable.

                                                                                          Visa is, to me, a single centralized thing. That makes the singular-Visa model of Visa the truth, the actual reality. Humans define what is real. And we have a basically fixed cognitive capacity. Abstractions are how we extend that cognitive capacity to higher-order outcomes. And abstractions require a minimum set of trust assumptions to be useful and non-leaky. Centralization is effectively the only way to establish those trust invariants.

                                                                                          1. 2

                                                                                            Sounds like you’re saying that humans are lazy and apathetic, they don’t think of the non-immediate consequences of their actions, and will just reach for the most comfortable solution regardless… And to some extent, you’re right. But is it a reason to give up? I’m okay with you being a cynic, but I’m not ready to sink with you just yet.

                                                                                            It’s interesting that you brought up Visa specifically. First, let’s get the technical point out of the way:

                                                                                            If I fat-fingered the recipient of my payment, it’s absolutely necessary that I can petition a well-defined, singular, authoritative —

                                                                                            1. I think you mean something like a judge.
                                                                                            2. You could ask the same of your payment provider even if it was much smaller than Visa. Banking delays are there for this very reason, in fact.

                                                                                            Now, it turns out the Visa/Mastercard duopoly is strangling some businesses and non-profits so hard that they’re effectively banning them. One casualty is a good chunk of the porn business. We could be tempted to say good riddance to this immoral den of women’s exploitation, but shouldn’t this be discussed democratically? Another casualty is Wikileaks, the most impactful journalistic organisation of the century, responsible for countless revelations in the entire world. (By the way, Assange is slowly dying at the hand of the USA and England, because of his Pulitzer-worthy work.)

                                                                                            It seems to me that a sufficiently diverse set of payment processing offers (in other words a decentralised payment system), is a necessary component of healthy democracies, without which speech isn’t as free as it should. For similar reasons, a sufficiently diverse set of internet hosting platforms is just as necessary.

                                                                                            I concede that there are forces for centralisation. They’re just not overwhelming. At the very least, there is such a thing as anti-trust laws and forcibly splitting companies. Preventing and reversing excessive centralisation may be a hassle, or even difficult, but the alternative is nothing less than the death of democracy and freedom.

                                                                                            I’m not quite ready to let that happen.

                                                                                            1. 2

                                                                                              Now, it turns out the Visa/Mastercard duopoly is strangling some businesses and non-profits . . .

                                                                                              You touch on an important dimension of the problem space which deserves attention. This should be fixed, I agree. But you have to address stuff like this while maintaining the essential properties of the underlying system(s). If your solution to porn businesses not being able to accept Visa payments means users who get defrauded of some payment or fat-finger a transfer or forget their master key or whatever don’t have any practical resource, then that’s not a solution, it’s a regression.

                                                                                              I think what this boils down to is a disconnect on the nature of large-scale systems as they exist today. The fundamental properties of these large-scale systems are the result of thousands of years of evolution, thousands of years of toil and experimentation and course correction. I’m no advocate of the status quo for it’s own sake, but I’m definitely an advocate for understanding the history and rationale for something before advocating change — a Chesteron’s Fence maxi, let’s say.

                                                                                              Why is the centralization fence in the field? Because it’s the natural outcome for large-scale systems. It’s not pathological, it’s inevitable, and Actually Good™. Humans have fixed cognitive capacity, scaling that capacity non-linearly requires abstraction, effective (non-leaky) abstractions require authoritative definitions, and authority requires centralization. Not complicated.

                                                                                              1. 1

                                                                                                If your solution to porn businesses not being able to accept Visa payments means users who get defrauded of some payment or fat-finger a transfer or forget their master key or whatever don’t have any practical resource

                                                                                                That’s a big if. If I recall correctly, users already had that kind of recourse before everything got centralised that way.

                                                                                                Humans have fixed cognitive capacity, scaling that capacity non-linearly requires abstraction, effective (non-leaky) abstractions require authoritative definitions, and authority requires centralization.

                                                                                                You lost me at the very last word. The kind of centralisation you should speak of here is a standard, either formal or emergent. One standard everyone agrees on, one standard to save everyone their cognitive resources, yet you can have many providers.

                                                                                                Some of the best examples I can think of are IP, TCP, UDP, and BitTorrent. One standard, many implementations, decentralised network. You get the cognitive benefits of centralisation without paying for it with a monopoly.

                                                                                                The fundamental properties of these large-scale systems are the result of thousands of years of evolution, thousands of years of toil and experimentation and course correction.

                                                                                                Evolution remains a mindless process that doesn’t have the same goals as us humans. I wouldn’t so readily treat it as an authoritative source.

                                                                                                Why is the centralization fence in the field? Because it’s the natural outcome for large-scale systems. It’s not pathological, it’s inevitable, and Actually Good™.

                                                                                                Standards are good. Monopolies… my political opinion is that they’re bad. And interestingly, we can observe diseconomies of scale too. We’ve seen for instance different branches of the same company lobby governments for opposite bills (I recall one example with Universal).

                                                                                                Also, how such huge conglomerate rise isn’t all that pretty. Though there is such a thing as “winner takes all”, once companies exceed a certain size they tend to squash competition with things other than efficiency or serving the customer better. They buy competitors, prevent interoperability, lobby for regulations small players can’t follow… it’s not exactly the free and unbiased competition we’ve been sold.

                                                                                                1. 1

                                                                                                  Some of the best examples I can think of are IP, TCP, UDP, and BitTorrent. One standard . . .

                                                                                                  These are protocols. Protocols are things defined well below the level of authorities understood by users. No human being gives a shit about TCP or BitTorrent, they care about the things that those things provide.

                                                                                                  Following…

                                                                                                  The kind of centralisation you should speak of here is a standard, either formal or emergent. One standard everyone agrees on, one standard to save everyone their cognitive resources, yet you can have many providers.

                                                                                                  Standards, providers, these things are details which are irrelevant to users. A user expressing the concept of Youtube doesn’t care about any of these things. Youtube is a single conceptual thing, invariant to delivery, protocol, standard, etc. It absolutely doesn’t matter how it’s resolved (via DNS, etc.) or how it’s served (via HTTP, HTTPS, QUIC, etc.) or anything else. It is one abstract thing, existing above those details.

                                                                                                  The thing the users of the system care about is the higher level, abstract concept of what they express. This is definitionally singular, and definitionally requires authority, which in turn requires centralization. There’s no avoiding this. You can’t decentralize a definition of Youtube.

                                                                                                  1. 1

                                                                                                    The thing the users of the system care about is the higher level, abstract concept of what they express.

                                                                                                    I want to see an internet video. Let me click on the button that let me see internet videos. I want to bypass my country’s Great Firewall. Let me click on the button that let me bypass that firewall. Got it.

                                                                                                    This is definitionally singular,

                                                                                                    One button to rule them all. Well, except users around the world won’t click on the same button. They’ll click on their own instance of the button, and depending on culture, languages, personal preference, or disabilities, the ideal button will be a little different. Not exactly singular.

                                                                                                    What is singular is the idea that I get access to the same stuff as everyone else. But that doesn’t mean everyone is hosted by the same company. It only means we’re all accessible through the same network.

                                                                                                    The Tor network is interesting in this respect, because to reliably give everyone access to the same network (untainted by some Great Firewall), it has to be decentralised. There’s basically one Tor Browser (one button to rule them all I guess), but there are many nodes.

                                                                                                    and definitionally requires authority, which in turn requires centralization.

                                                                                                    Yes, some authority needs to define the protocols, someone needs to write server/client or peer code, and most people will just use the most popular one. But then again, this protocol and code only needs to give access to the same network.

                                                                                                    There’s no avoiding this.

                                                                                                    Indeed, there’s no avoiding a single network.

                                                                                                    You can’t decentralize a definition of Youtube.

                                                                                                    You’re not getting away with a tautology. YouTube is several things, really:

                                                                                                    • A video hosting service
                                                                                                    • A search & recommendation service
                                                                                                    • An ad network.

                                                                                                    Unfortunately it’s hard to escape a centralised search & recommendation service, because those are very capitalistic (they require a significant investment up front). I hate that, but I do have to concede this point.

                                                                                                    The ad network… well it’s not that centralised even on YouTube. Many content creators disable ads, or have their videos demonetised, so they rely on “sponsors” instead, and those are quite varied. There are also donation platforms, and we do have very few of them, but I believe the network effects for them is not as strong as the network effects borne out of the need to… just click the button to see internet videos.

                                                                                                    The video hosting service however doesn’t need to be centralised in a single company. BitTorrent is proof of this. People need to all be on the same network, they need to be reachable and searchable for sure, but hosting itself (and the rules for hosting) can easily be very decentralised. See PeerTube.

                                                                                                    Even the concept of subscribing to a channel has already been demonstrated (and was widely used) with RSS (and then Atom). Even today I get the occasional request that I update my feeds, even though RSS/Atom are supposed to be “dead”. It wouldn’t take much, technically, to write a unified PeerTube client that let user access to all PeerTube servers and watch videos like everything was hosted in the same place — even though it’s only the same network.

                                                                                                    1. 1

                                                                                                      I want to see an internet video.

                                                                                                      You want to see an internet video — with a specific identifier X.

                                                                                                      The identifier X must be, for all users of the system, no matter where they are, (a) humane (e.g. can be communicated over a phone call); (b) deterministic (i.e. youtube.com/abc cannot ever be one video for me and a different video for you); and (c) available (i.e. fetching youtube.com/abc must succeed 99%+ of the time).

                                                                                                      These properties are table stakes. A system which does not satisfy them is non-viable. HTTP URLs work (barely). BitTorrent magnets, IPFS hashes, etc. do not.

                                                                                                      Search and recommendation systems translate abstract user query Q to identifier X. Ad networks decorate requests for identifier X with stuff that generates revenue for stakeholders S1, S2, etc. But it all boils down to well-defined resources with the above properties.

                                                                                                      The network is an implementation detail, ultimately irrelevant. The user experience is the thing that matters.

                                                                                                      I agree that it is theoretically possible for any kind of network, including a decentralized network, to solve this problem. But as far as I’m aware, it is factually impossible to assert an authoritative and verifiable definition of a qualifying identifier X, and practically impossible to offer acceptable levels of availability for the content behind that identifier, without centralization.

                                                                                                      If I’m wrong, tell me why! Show me a counter example. I’m honestly eager to be corrected.

                                                                                                      1. 1

                                                                                                        It seems to me YouTube itself fails to uphold the table stakes you speak of. It blocks videos, sometimes in a country by country basis. It arbitrarily unsubscribes people from channels without asking either the subscriber or the author. It has lots of rules for publications that stops authors from publishing many kinds of videos that would otherwise be perfectly legal in the US or the EU. So before we even think of referring to a video, we should think of a system that can reliable publish it.

                                                                                                        HTTP URLs work (barely). BitTorrent magnets, IPFS hashes, etc. do not.

                                                                                                        YouTube uses random unique identifiers, same as magnet links and IPFS hashes. I mean look at this: https://www.youtube.com/watch?v=l1ujHfWoiOU. Have you ever dictated “l1ujHfWoiOU” over the phone? I tried to, and without a nice monospace font, I never know whether the first character is an uppercase I or a lowercase l. And without those identifiers you do not reliably access even publicly available videos, because search is (i) finicky, (ii) location sensitive, and (iii) some videos are explicitly excluded from search queries (and not just at he behest of the author).

                                                                                                        The network is an implementation detail, ultimately irrelevant.

                                                                                                        Indeed. I care most about the governance of the network.

                                                                                                        That’s the ultimate problem with YouTube. They could use BitTorrent under the hood to cut down their server costs to almost nothing at all, it wouldn’t necessarily change their recommendation system, search queries, or subscription model. People could probably give direct access to their videos even if Alphabet doesn’t want them to, but then we’re back to unreadable hashes.

                                                                                                        Still, not all networks can be governed the same way. Good luck decentralising the governance of a single data centre for instance. If we want people to be able to talk to each other without some corporation deciding for everyone, it has to be physically decentralised to begin with. Even here for instance, we’re talking in a public forum handled by a handful of moderators. Bless them for their work, but if one of us gets out of line, we’ll quickly get a reminder that this is not our turf. We wouldn’t have the same restrictions over email, or our respective blogs.

                                                                                                        (I’m not saying centralisation is all bad: we gather here in Lobsters mostly of our own accord, and a central-ish governance for the sake of moderation makes a lot of sense here. There are other, more popular forums out there if this one doesn’t suit us. The advantages of moderation and curation are real, and the price is very tame when this is not a near-monopoly like YouTube.)

                                                                                                        But as far as I’m aware, it is factually impossible to assert an authoritative and verifiable definition of a qualifying identifier X,

                                                                                                        Unless that identifier is a hash. Can’t be manually copied, so you need computer aided copy&paste, the ability to click on the link, or a QR-code. Note however that with unreadable hashes come one little benefit: they can include a certificate or the public key of the publisher. So as long as you get them from a trusted source (a geek friend who is recommending some author, or the bank teller handing you a card with a QR-code on it), what you get is even more authoritative and verifiable than a regular Web URL (which by the way is vulnerable to typo squatting).

                                                                                                        It’s just… not readable by humans.

                                                                                                        and practically impossible to offer acceptable levels of availability for the content behind that identifier, without centralization.

                                                                                                        There’s something here that I wanted to say quite a long ago in this thread: the availability of a decentralised system is very different from that of a centralised one. Let’s keep the YouTube analogy:

                                                                                                        • A single video may not be available.
                                                                                                        • A single channel may not be available.
                                                                                                        • YouTube itself may not be available.

                                                                                                        As a whole, it is indeed unacceptable for YouTube to have less than 4 nines of availability. If it goes down for more than 0.01% of the time, something is very wrong. But lucky us, decentralised networks basically never go down that way: you’d have to convince every node to shut down, and if they’re diverse enough not even a botched update will do that.

                                                                                                        For a single channel though? 3 nines (going down 0.1% of the time) is more than enough most of the time, and for low-stakes stuff we could even go down to 2 nines. I could do it with self hosting at home. My grandma could ask a 3rd party hosting provider, similar to what I do for my website.

                                                                                                        And that’s without distributed backup. Sure, applying a distributed backup layer manually is likely to be a pain, or even impossible, but if it’s implemented as part of the hosting solution… I mean, if nodes dedicate a reasonable percentage (likely between 5% to 20%) to error correction codes for other people’s videos, 2 nines availability for any single channel can easily turn into 3 or 4 nines as if by magic.

                                                                                                        1. 1

                                                                                                          It seems to me YouTube itself fails to uphold the table stakes you speak of. It blocks videos, sometimes in a country by country basis. It arbitrarily unsubscribes people from channels without asking either the subscriber or the author. It has lots of rules for publications that stops authors from publishing many kinds of videos that would otherwise be perfectly legal in the US or the EU. So before we even think of referring to a video, we should think of a system that can reliable publish it.

                                                                                                          I’m not sure why you’d expect any system to operate without regard to relevant laws or regulations. If a video is removed from a hosting provider for e.g. copyright infringement or whatever, that’s (hopefully non-controversially) unrelated to availability in the sense that we’re talking about here.

                                                                                                          it is factually impossible to assert an authoritative and verifiable definition of a qualifying identifier X,

                                                                                                          Unless that identifier is a hash . . . [which] can include a certificate or the public key of the publisher. So as long as you get them from a trusted source (a geek friend who is recommending some author, or the bank teller handing you a card with a QR-code on it), what you get is even more authoritative and verifiable than a regular Web URL (which by the way is vulnerable to typo squatting).

                                                                                                          This notion of trust in the authenticity of the hash — some out-of-band or sidechannel mechanism by which information can be personally or individually verified — doesn’t scale beyond Dunbar’s number. Even if it were conceivable to suggest users would only interact with URLs sent to them from their cryptographically verifiable inner sanctum of close friends (it isn’t) you can’t build anything but trivial systems out of that foundation. Anything that serves meaningful numbers of human users needs to have mechanisms for delegation of trust.

                                                                                                          Assume for the sake of the argument that you can trust the hash (you can’t, but). And say that hash includes some a signature over the data. For this to provide meaningful assurances, you need, at a minimum, a trusted source of truth for the authenticity of the signature, and reliable defenses against MITM attacks of the hash and the content. It’s fractal trust assumptions, all the way down.

                                                                                                          And then! That hash still needs to be resolved to a physical address from which you can read the actual content. What system does that resolving? How does it work? Specifically, can that system provide anything resembling acceptable latency without trust delegation? (Spoiler: nope, you need caching, caching requires trust, etc. etc.)

                                                                                                          the availability of a decentralised system is very different from that of a centralised one . . . As a whole, it is indeed unacceptable for YouTube to have less than 4 nines of availability. If it goes down for more than 0.01% of the time, something is very wrong. But lucky us, decentralised networks basically never go down that way: you’d have to convince every node to shut down, and if they’re diverse enough not even a botched update will do that.

                                                                                                          Availability isn’t a theoretical measure of content delivered eventually, it’s a measure of actual user experience. If I open my DecentralWeb browser and paste in a trusted hash, and I don’t see the content I expect in something like 5 seconds at the most, then that content is not available. And if I experience that more than just sporadically, then the DecentralWeb network is down.

                                                                                                          It doesn’t matter if a single node in some remote EC2 region, or on some ADSL connection in Marseilles, has the content I’m looking for, if I can’t actually fetch it in a reasonable timeframe. And you just don’t get that sort of availability, at non-trivial scale, out of consumer network appliances attached to home internet connections joining [permissionless] decentralized networks.

                                                                                                          2 nines . . . I could do it with self hosting at home

                                                                                                          I think you’re making some simplifying assumptions. Sure, if you host a video that gets 1 view a day, nothing really matters. But availability is measured invariant to stuff like request load or internet weather or etc. If you make some video that goes viral, and suddenly you’re seeing 10k RPS for the thing, your home internet connection will factually not get you any nines of reliability ;)

                                                                                                          1. 1

                                                                                                            I’m not sure why you’d expect any system to operate without regard to relevant laws or regulations.

                                                                                                            My point is, YouTube goes well above and beyond any laws and regulation. It arbitrarily unsubscribes random people from channels. There’s no law that forces it to. It arbitrarily unreferences some videos even though they’re legal. The appeal process for takedown notices doesn’t leave room for any kind of fair use. That kind of thing wouldn’t affect a decentralised system where each author has to be contacted directly for possible unlawful content, where each user would be subscribed to each author by an RSS/Atom feed or similar.

                                                                                                            This notion of trust in the authenticity of the hash — some out-of-band or sidechannel mechanism by which information can be personally or individually verified — doesn’t scale beyond Dunbar’s number.

                                                                                                            Correct… and incorrect.

                                                                                                            Users obviously cannot meaningfully trust more than 200 sources directly. In practice they can only track the most important website they visit, their family and friends, the physical places they go to that also have an online presence.

                                                                                                            On the other hand, it’s very possible to be directly trusted by millions of users. Take banks for instance. People usually go physically to the bank. When they do so, the clerk can give them a little card with a QR-code on it, that contains every information needed to contact & authenticate the bank. Some URL equivalent (except it doesn’t need to be readable), and the public key of the bank’s certificate authority. The source of trust here is “I went to the Bank and I trust that the clerk there gave me a genuine card”. That’s not perfect obviously, but it should be better than the web’s PKI already. More importantly though, people can use the physical interactions they are used to to establish trust — or distrust.

                                                                                                            Availability isn’t a theoretical measure of content delivered eventually, it’s a measure of actual user experience.

                                                                                                            Of course it is. What made you think I was speaking of anything other than the user experience?

                                                                                                            • YouTube goes down for 1% of the time? I get nothing 3 days a year or so.
                                                                                                            • YouTube channels go down 1% of the time? I get 99% of what I want to see all the time.

                                                                                                            Over-simplifying of course, but you can already see that this is not the same user experience. Hence my conjecture that the temporary availability of any given channel is much less important than the availability of YouTube as a whole. Nobody in their right mind would say YouTube is down because 2 of their favourite channels are unavailable this day. The same reasoning apply to a distributed system.

                                                                                                            Sure, if you host a video that gets 1 view a day, nothing really matters. But availability is measured invariant to stuff like request load or internet weather or etc. If you make some video that goes viral, and suddenly you’re seeing 10k RPS for the thing

                                                                                                            Doesn’t sound like such a big problem to be honest. Assuming a request and its response take up to a Kilobyte, we’re talking something like 80 Megabits per second in both directions. It’s high for sure, but not out of reach of most fibre connections I believe. And if people get the equivalent of a magnet link instead of querying my server directly I won’t get anywhere close to 10k RPS.

                                                                                                            Even if we can’t avoid some centralisation there, remember that the entirety of the Pirate Bay was never more than 4 server racks. The most popular torrent tracker in the world, with website, ads, search, torrent files & magnet links, all in 4 racks.

                                                                                                            1. 1

                                                                                                              YouTube . . . arbitrarily unsubscribes random people from channels.

                                                                                                              (YouTube.com has different availability vs. a given YouTube channel)

                                                                                                              I don’t believe either of these claims are true. They’re facially nonsensical at a technical level, and would be self-subversive to YouTube as an organization if they were enshrined as any kind of policy. What makes you think otherwise? Links or etc.?

                                                                                                              Doesn’t sound like such a big problem to be honest. Assuming a request and its response take up to a Kilobyte . . .

                                                                                                              Sustained 10k RPS from arbitrary clients over the internet was my random example, but in fact that’s (a) orders of magnitude less than the traffic levels that viral content triggers in this day and age; and even ignoring that (b) it’d be bottlenecked not by available bandwidth but by the receiving node’s network stack. A reasonably optimized application (e.g. a compiled language written by a senior engineer) running on server-class hardware (e.g. Xeon) in a well-connected datacenter (e.g. redundant and 1-2 hops from a backbone) can reasonably be expected to serve 10k RPS. A consumer device on a home connection satisfies none of these conditions.

                                                                                                              Also, self-hosting doesn’t mean hosting the hash or magnet link for a bit of content that’s served by other peers in a network, it means serving the content directly. But okay, let’s say that when grandma “self-hosts” a video, she’s actually copying the video to an arbitrary number of unknown peers. The immediate question then becomes: how does she delete a video she posted accidentally? This isn’t a nice-to-have, it’s table stakes.

                                                                                                              Some URL equivalent (except it doesn’t need to be readable), and the public key of the bank’s certificate authority. The source of trust here is “I went to the Bank and I trust that the clerk there gave me a genuine card”. That’s not perfect obviously, but it should be better than the web’s PKI already. More importantly though, people can use the physical interactions they are used to to establish trust — or distrust.

                                                                                                              How is the the delegation of trust performed when one loads a URL provided by a bank teller via a piece of paper containing a QR code any different than the delegation of trust performed when one types www.bankofamerica.com into the URL bar of their web browser?

                                                                                                              1. 1

                                                                                                                YouTube . . . arbitrarily unsubscribes random people from channels.

                                                                                                                I don’t believe either of these claims are true

                                                                                                                I don’t have hard evidence, but I do have several account of various channels complaining about various things, including shadowbanning (popular content not showed in the recommendations), blatant biases in the recommendation system (not necessarily for nefarious reasons but still) and yes, involuntary un-subscriptions. Here’s one example of some thing a content creator might complain about.

                                                                                                                But even if I’m wrong (I reckon I’m not very confident in that particular claim), YouTube indisputably does strike or demonetises individual videos and ban whole channels based on things that (i) go beyond the law, and (ii) go beyond their own terms of service. Not necessarily on purpose, but at that scale they have to make lots of mistakes. And we’ve all heard stories about their not-so-fine support and appeal process.

                                                                                                                (YouTube.com has different availability vs. a given YouTube channel)

                                                                                                                I don’t believe either of these claims are true

                                                                                                                It wasn’t a statement of fact, it was an example. What if we had partial availability? Doesn’t make sense for YouTube of course, but it does make sense for any distributed system. I mean the Web is not down just because 5% of the websites decided to shut themselves down in protest, is it?

                                                                                                                Sustained 10k RPS from arbitrary clients over the internet was my arbitrary example, and in fact is […] bottlenecked not by available bandwidth but by the receiving node’s network stack.

                                                                                                                Yeah, this I did gloss over this problem even though I was aware of it. There’s one thing though, we have some evidence suggesting that the average Linux network stack is slow. My guess here is that if we were serious about selling low-cost appliances tailored for network loads, we could handle this 10k RPS on much less powerful hardware than our current server.

                                                                                                                It’s not easy to solve, mind you. This is a “company builds new hardware with new OS and gets successful” levels of hard. Technically feasible, good luck with the economics and the network effects. I don’t expect it in the foreseeable future.

                                                                                                                Also note that viral content is rare. It only looks frequent because we see viral content all the time thanks to availability bias. (Same thing for the lottery: we basically never win, but we often hear the winners.)

                                                                                                                Also, self-hosting doesn’t mean hosting the hash or magnet link for a bit of content that’s served by other peers in a network, it means serving the content directly.

                                                                                                                It’s a UX thing. If we can not-really-host content while giving a sufficient illusion that we are hosting it, that’s enough for me. Though in this case I was thinking more along the lines of providing a peer in addition to the magnet link. You do host the file, but if it goes viral the load is still distributed among the peers that are currently downloading it.

                                                                                                                The immediate question then becomes: how does she delete a video she posted accidentally? This isn’t a nice-to-have, it’s table stakes.

                                                                                                                Those table stakes don’t exist even on YouTube. If someone has copied the video, it’s too late, the cat’s out of the bag. You might get lucky and delete the video before someone else sees the magnet link, same as YouTube, but beyond that you can only hope that people will lose interest or be decent enough to stop exchanging your video.

                                                                                                                Defaults matter a lot here. Watching a YouTube video doesn’t retain it by default. Downloading a video on BitTorrent does, making reuploads so much easier. But nothing stops things like PeerTube from retaining videos only long enough to make peering work. If retaining the videos require users to click the “save” button, most won’t.

                                                                                                                1. 2

                                                                                                                  We have now fully departed the material conversational plane and transcended to a higher dimension, speaking both directly at and entirely past each other, a simultaneous expression and destruction of the self. Glory and pity to us both, amen.

                                                                                                                  1. 2

                                                                                                                    Well, we tried. This still got me thinking, so thank you for having stuck here.

                                                                        1. 12

                                                                          Programmers have a long and rich history with C, and that history has taught us many lessons. The chief lesson from that history must surely be that human beings, demonstrably, cannot write C code which is reliably safe over time. So I hope nobody says C is simple! It’s akin to assembly, appropriate as a compilation target, not as an implementation language except in extreme circumstances.

                                                                          1. 9

                                                                            Which human beings? Did history also teach us that operating a scalpel on human flesh cannot be done reliably safe over time?

                                                                            Perhaps the lesson is that the barrier of entry for an engineering job was way higher 40 years ago. If you would admit surgeons to a hospital after a “become a gutt-slicer in four weeks” program, I don’t think I need to detail what the result would be.

                                                                            There’s nothing wrong with C, just like there’s nothing wrong with a scalpel. We might have more appropriate tools for some of its typical applications, but iC s still a proven useful tool.

                                                                            Those who think their security burns will be solved by a gimmick such as changing programming language, are in for a very unpleasant surprise.

                                                                            1. 22

                                                                              Perhaps the lesson is that the barrier of entry for an engineering job was way higher 40 years ago

                                                                              Given the number of memory safety bugs that have been found in 40-year-old code, I doubt it. The late ‘90s and early 2000s exposed a load of these bugs because this C code written by skilled engineers was exposed to a network full of malicious individuals for the first time. In the CHERI project, we’ve found memory safety bugs in code going back to the original UNIX releases. The idea that there was some mythical time in the past when programmers were real men who never introduced security bugs is just plain wrong. It’s also a weird attitude: a good work an doesn’t blame his tools because a good work an chooses good tools. Given a choice between a tool that can be easily operated to produce good results and one that, if used incredibly carefully, might achieve the same results, it’s not a sign of a good engineer to choose the latter.

                                                                              1. 4

                                                                                Given the number of memory safety bugs that have been found in 40-year-old code, I doubt it.

                                                                                Back then, the C programmers didn’t know about memory safety bugs and the kind of vulnerabilities we have since two decades. Similar, Javascript and HTML are surely two programming languages which are somewhat easier to write than C and doesn’t suffer from the same class of vulnerabilities. However, 20 years ago people wrote code in these two languages that suffer from XSS and other web based vulns. Heck, XSS and SQLi is still a thing nowadays.

                                                                                What I like about C is that it forces the programmer to understand the OS below. Writing C without knowing about memory management, file descriptors, processes is doomed to fail. And this is what I miss today and maybe @pm in their comment hinted at. I conduct job interviews with people who consider themself senior and they only know the language and have little knowledge about the environment they’re working in.

                                                                                1. 4

                                                                                  Yes, and what we have now is a vast trove of projects written by very smart programmers, who do know the OS (and frequently work on it), and do know how CPUs work, and do know about memory safety problems, and yet still cannot avoid writing code that has bugs in it, and those bugs are subsequently exploitable.

                                                                                  Knowing how the hardware, OS (kernel and userspace), and programming language work is critical for safety or you will immediately screw up, rather than it being an eventual error.

                                                                                  People fail to understand that the prevalence of C/C++ and other memory unsafe languages has a massive performance cost: ASLR, Stack and heap canaries, etc and then in hardware: PAC, CFI, MTE, etc all have huge performance costs in modern hardware, are all necessary solely due to the need for the platform to mitigate the terrible safety of the code being run. That’s now all sunk cost of course: if you magically shifted all code today to something that was memory safe, the ASLR and various canaries costs would still be there - if you were super confident your OS could turn ASLR off, and you could compile canary free, but the underlying hardware is permanently stuck with those costs.

                                                                                  1. 2

                                                                                    Forcing the programmer to understand the OS below could (and can) happen languages other than C. The main reason it doesn’t happen is that OS APIs, while being powerful, are also sharp objects that are easy to get wrong (I’ve fixed bugs in Janet at the OS/API level, I have a little experience there), so many languages that are higher level end up with wrappers that help encode assumptions that need to not be violated.

                                                                                    But, a lot of those low level functions are simply the bottom layer for userland code, rather than being The Best Possible Solution as such.

                                                                                    Not to say that low level APIs are necessarily bad, but given the stability requirements, they accumulate cruft.

                                                                                  2. 1

                                                                                    The programmer and project that I have sometimes used as a point of comparison is more recent. I’m now about the same age that Richard Hipp was when he was doing his early work on SQLite. I admire him for writing SQLite from scratch in very portable C; the “from scratch” part enabled him to make it public domain, thus eliminating all (or at least most) legal barriers to adoption. And as I mentioned, it’s very portable, certainly more portable than Rust at this point (my current main open-source project is in Rust), though I suppose C++ comes pretty close.

                                                                                    Do you have any data on memory safety bugs in SQLite? I especially wonder how prone it was to memory safety bugs before TH3 was developed.

                                                                                  3. 7

                                                                                    Did history also teach us that operating a scalpel on human flesh cannot be done reliably safe over time?

                                                                                    I think it did. It’s just that the alternative (not doing it) is generally much much worse.

                                                                                    There’s nothing wrong with C, just like there’s nothing wrong with a scalpel.

                                                                                    There is no alternative to the scalpel (well, except there is in many circumstances and we do use them). But there can be alternatives to C. And I say that as someone who chose to write a new cryptographic library 5 years ago in C, because that was the only way I could achieve the portability I wanted.

                                                                                    C does have quite a few problems, many of which could be solved with a pre-processor similar to CFront. The grammar isn’t truly context free, the syntax has a number of quirks we have since learned to steer clear from. switch falls though by default. Macros are textual instead of acting at the AST level. Everything is mutable by default. It is all too easy to read uninitialised memory. Cleanup could use some more automation, either with defer or destructors. Not sure about generics, but we need easy to use ones. There is enough undefined behaviour that we have to treat compilers like sentient adversaries now.

                                                                                    When used very carefully, with a stellar test suite and sanitisers all over the place, C is good enough for many things. It’s also the best I have in some circumstances. But it’s far from the end game even in its own turf. We can do better.

                                                                                    1. 2

                                                                                      And I say that as someone who chose to write a new cryptographic library 5 years ago in C, because that was the only way I could achieve the portability I wanted.

                                                                                      I was wondering why the repo owner seemed so familiar!

                                                                                    2. 3

                                                                                      Those who think their security burns will be solved by a gimmick such as changing programming language, are in for a very unpleasant surprise.

                                                                                      I don’t think that moving from a language that e.g. permits arbitrary pointer arithmetic, or memory copy operations without bounds checking, to a language that disallows these things by construction, can be reasonably characterized as a gimmick.

                                                                                      There’s nothing wrong with C, just like there’s nothing wrong with a scalpel.

                                                                                      This isn’t a great analogy, but let’s roll with it. I think it’s uncontroversial to say that neither C nor scalpels can be used at a macro scale without significant (and avoidable) negative outcomes. I don’t know if that means there is something wrong with them, but I do know that it means nobody should be reaching for them as a general or default way to solve a given problem. Relatively few problems of the human body demand a scalpel; relatively few problems in computation demand C.

                                                                                      1. 2

                                                                                        That’s a poor analogy.

                                                                                        What we would consider “modern” surgery had a low success rate, and a high straight up fatality rate.

                                                                                        If we are super generous, let’s say C is a scalpel. In that case we can look at the past and see a great many deaths were caused by people using a scalpel, long after it was established that there was a significant differences in morbidity when comparing a scalpel, to a sterilized scalpel.

                                                                                        What we have currently is a world where we have C (and similar), which will work significantly better than all the tools the preceded it, but is also very clearly less safe than any modern safe language.

                                                                                    1. 1

                                                                                      If anyone wants to see another example of this in practice I had to build an SSH port forwarder for DataStation so that you could connect to databases/web servers running on remote servers.

                                                                                      func (ec EvalContext) withRemoteConnection(si *ServerInfo, host, port string, cb func(host, port string) error) error {
                                                                                      

                                                                                      https://github.com/multiprocessio/datastation/blob/main/runner/ssh.go

                                                                                      If it’s helpful to anyone trying to do this, steal any code/ideas in there!

                                                                                      1. 7
                                                                                        localConn, err := net.Listen("tcp", "localhost:0")
                                                                                        ...
                                                                                        localPort := localConn.Addr().(*net.TCPAddr).Port
                                                                                        

                                                                                        Using net.ListenTCP instead of net.Listen avoids the need for the type assertion, and the concomitant runtime risks.

                                                                                        errC := make(chan error)
                                                                                        // Local server
                                                                                        go func() { ... errC <- <error> ... } 
                                                                                        
                                                                                        ...
                                                                                        
                                                                                        localPort := ...
                                                                                        cbErr := cb("localhost", fmt.Sprintf("%d", localPort))
                                                                                        if cbErr != nil {
                                                                                        	return cbErr
                                                                                        }
                                                                                        

                                                                                        If you hit the return cbErr code path, the function will return without ensuring the local server goroutine has terminated, which is a potential leak. Easy fix: move the cb block before the go func block.

                                                                                        	select {
                                                                                        	case err = <-errC:
                                                                                        		... return ...
                                                                                        	default:
                                                                                        		return nil
                                                                                        	}
                                                                                        }
                                                                                        

                                                                                        The default case here means that withRemoteConnection always return immediately. It will return a nil error by default, and a non-nil error only if (a) the local server goroutine you spawned previously is scheduled by the Go scheduler somehow between the go call and this select block, and (b) that code produces an outcome and sends it to the errC chan before the scheduler executes this block. In any other case, that local server goroutine will block forever on its send to the (unbuffered) errC, which leaks the goroutine. I suppose this isn’t your intent?

                                                                                        1. 2

                                                                                          Thank you for the notes!

                                                                                        1. 4

                                                                                          I teach classes on Go at Google, but I am not a language author so don’t take my word for it.

                                                                                          I have found it pays to always pass pointers to structs, never copy. It’s very easy to pass a struct to a function with a copy, mutate the struct, and then find out (much) later that the original doesn’t mutate because you mutated the copy. Many hours can be wasted debugging this.

                                                                                          There’s little value in copying, Go doesn’t have consts and pretending it is immutable will get you into trouble. I have seen some “experts” saying to use copies to try and create immutability, but this is trying to shoehorn something into the language that doesn’t exist. It makes code very hard to read and reason about. If the compiler can’t help you – which the Go compiler can’t because there’s no const keyword – then you shouldn’t try it. Note if you pass a copied struct but that struct itself contains a field that’s a pointer, then that field isn’t immutable and so you’re still in the same spot.

                                                                                          1. 3

                                                                                            I have found it pays to always pass pointers to structs, never copy.

                                                                                            +1 to your intuition here — I always default to operating on struct pointers, and only opt-in to struct values as special cases.

                                                                                            Go doesn’t have consts and pretending it is immutable will get you into trouble. I have seen some “experts” saying to use copies to try and create immutability, but this is trying to shoehorn something into the language that doesn’t exist.

                                                                                            +1 to your judgment here, too — I’d be very surprised to see any Go “expert” suggest such a thing!

                                                                                            1. 2

                                                                                              Can confirm I went to a workshop at Gophercon where the “copy for immutability” was espoused. Names withheld to protect the guilty :)

                                                                                              1. 1

                                                                                                :o

                                                                                          1. 4

                                                                                            FWIW, to prevent the bug where a = b is slow for big types, Google’s C++ style guide used to mandate DISALLOW_COPY_AND_ASSIGN (which used to be DISALLOW_EVIL_CONSTRUCTORS I think) on all types (most types?)

                                                                                            Instead you would do something like a = b.Clone() I believe

                                                                                            Looks like that’s been gone for awhile in favor of C++ 11 stuff, which I don’t really like:

                                                                                            https://google.github.io/styleguide/cppguide.html#Copyable_Movable_Types

                                                                                            A lot of good software was written in that style, but it has grown bureaucratic over time, and as the C++ language evolved

                                                                                            1. 6

                                                                                              Indeed, C++ also makes it easy to create accidental copies, because it’s kind of the default. Relevant example:

                                                                                              for (const auto accidental_copy : collection) {}
                                                                                              

                                                                                              Rust is like you describe with Google’s style guide: No implicit copying, but a clone trait you can implement and call. It’s one of those things that make the language a bit harder, but mostly because it makes mistakes harder.

                                                                                              Even python seems to agree, in principle:

                                                                                              > python -c 'import this' | grep Explicit
                                                                                              Explicit is better than implicit.
                                                                                              
                                                                                              1. 8

                                                                                                Rust’s plain-data types can opt in to implicit copies (derive Copy). But it’s really nice to have no copies as the default.

                                                                                                Also ability to return a temporary read-only reference helps in cases like this, because you can give access to an element without copying it and without risk of the caller modifying it or being left with a dangling pointer.

                                                                                                1. 2

                                                                                                  You’re supposed to not derive Copy unless your type can be blitted (or just memcpy’d) rather than needing actual construction. I don’t know if implementing it when you’re not supposed to can break compiler assumptions (as I’ve observed Clone still gets called) but as a rule, it would avoid the worst of these slow copy cases.

                                                                                                  1. 2

                                                                                                    Yeah, Copy types are the exception that proves the rule. They aren’t a performance concern in the same way, because they can’t own a resource (because of that memcpy copyability: if it needs a destructor, it should need a custom clone function).

                                                                                                    Because this exception is really a sensible distinction, you can still search the codebase for “clone” and be confident you aren’t copying resources, which is what mostly matters, and is more readable than C++, where you have to delete operators to see if they are being called, which is easier said than done for non-custom types.

                                                                                                2. 4

                                                                                                  C++ got here via a very complex path. First, C had structures and structure assignment was defined as memcpy. Removing that would break compatibility with any C headers that contained functions or macros that do structure assignment. Then C++ decided that struct and class were equivalent except for the default visibility, which meant that the rule applied for any C++ object as well as any C structure. But some C++ classes needed to do non-trivial things (not just memcpy) on copy, and so added copy constructors and copy assignment.

                                                                                                  Smalltalk-derived languages assumed garbage collection and so required all objects to be allocated on the heap, but C++ did not want to do this (and couldn’t for C compatibility) and so came with the additional constraint that you had to be able to copy an object into space reserved by the caller of the copy operation (this includes copying objects into globals and onto the stack). This meant that you coudn’t implement something like clone(), where the return type depends on subclass relationship, you had to model these things as separate allocation and initialisation.

                                                                                                  This is further complicated by the fact that returning a structure is, in C++, modelled as copying the structure to the caller. For a long time, this copy has been optional (compilers may elide it - if you declare a local structure then return it then it will actually be created in the space that the caller passes down the stack). This was fun because copy constructors can have side effects and so it was observable in the abstract machine whether the compiler did this optimisation. C++17 made this mandatory, but this meant that any object that you wanted to return as an on-stack value needed to have a copy constructor.

                                                                                                  So now we’re in a state where you need copy constructors and you need them to exist by default and where deleting them breaks some language features. Now C++11 came along with auto types. C++ has syntactic modifiers to say ‘pointer-to-X and 'reference-to-X but nothing (outside of exciting type_traits things) to say bare-type-that-this-reference-thing-applies-to. If auto deduces T, then you can modify it to a reference by adding &, but if auto deduces T& then it’s hard to modify it back to T. This meant that auto had to default to deducing T, which meant that it was very easy to accidentally copy things. Compilers try to warn about this.

                                                                                                  Are also some interesting corner cases. You can make copy constructors explicit (so they won’t be invoked without explicit syntax saying that you meant to call them) but you can’t do the same for copy assignment. Herb’s new syntax cleans up a lot of this but the Rust approach (which doesn’t have any of the legacy baggage) is much better than the current state of C++.

                                                                                                  1. 1

                                                                                                    I actually am glad that auto usually gives T and not &T in these contexts - imagine how hard it would be to tell at a glance if a copy was being made or not (even if avoiding the copy requires an extra symbol in the soup).

                                                                                                  2. 3

                                                                                                    Yeah I definitely like opt-in better than opt-out, especially since code is generated! Too bad Go doesn’t have a mechanism for either one.

                                                                                                  3. 1

                                                                                                    It is really hard to create a type in Go where copies like this are slow, because almost all primitive Go types have reference semantics. In practice the only way is to define types containing arrays (not slices) of substantial size. I haven’t seen a lot of Go code that has e.g. type x struct with buf [10000]uintptr or whatever.

                                                                                                    1. 2

                                                                                                      Yeah that is an interesting difference, but I wouldn’t say “really hard” – since the OP had the problem, and someone else pointed to a comment where Russ Cox fixed the same problem in the Go standard library for a 40 byte struct, which was presumably all primitives


                                                                                                      Digression …. I think contrasting the languages is interesting:

                                                                                                      1. Rust: opt in to a = b being a copy for value types
                                                                                                      2. C++: opt out of a = b being a copy for value types
                                                                                                      3. Go: a = b is always a copy for value types

                                                                                                      Thinking about it a bit more, I think it’s a misfeature to reuse the same syntax for 2 different things

                                                                                                      1. a = b is a pointer assignment – O(1) with respect to the size of the type
                                                                                                      2. a = b is a value assignment – O(N) with respect to the size of the type

                                                                                                      e.g. in my head I have been making a big list of things where “syntax doesn’t match semantics”, and it hadn’t occurred to me that this is one

                                                                                                      I get your caveat that Go and C++/Rust structs aren’t equivalent though

                                                                                                      1. 1

                                                                                                        in my head I have been making a big list of things where “syntax doesn’t match semantics”,

                                                                                                        I’m interested: what else is on that list?

                                                                                                        1. 1

                                                                                                          A perf bug I have fixed repeatedly in other people’s Python code is:

                                                                                                          if item in L:  # linear search through list
                                                                                                            pass
                                                                                                          
                                                                                                          if item in D:  # dict or set membership
                                                                                                            pass
                                                                                                          

                                                                                                          If this is in a loop, then the first one is O(N^2), whereas the second is O(N), and I’ve seen that make scripts go from a 5 minute runtime to a second or so..

                                                                                                          So that is very similar to a = b, the syntax is too close for operations that mean different things. Most people just look at the syntax and don’t “visualize” what’s going on underneath, which is very different.


                                                                                                          A really trivial one that I remember someone having an extremely hard time with: static means 3 or 4 different things in C and C++:

                                                                                                          static void f();  
                                                                                                          static int x;
                                                                                                          

                                                                                                          A Java programmer simply couldn’t wrap his head around the fact that the annotation on f() means “local to this translation unit” / not exposed to the linker. It’s a gratuitous “pun” because they didn’t want to reserve a new keyword, which is extremely confusing.


                                                                                                          A lot of these syntactic conflicts are about shell. For example it took me a long time to realize that quoting means different things in different contexts in shell!

                                                                                                          echo $x
                                                                                                          echo "$x"
                                                                                                          echo hi > $x
                                                                                                          echo hi > "$x"
                                                                                                          a=$x
                                                                                                          a="$x"
                                                                                                          local a=$x
                                                                                                          local a="$x"
                                                                                                          

                                                                                                          I never wrote a blog post about that, but most shell experts won’t be able to tell you the difference between them. Most of all because common shells don’t even agree on what they mean :)


                                                                                                          Another example: People don’t know how to use the find command because its syntax doesn’t match what they are used to for expression semantics: https://www.oilshell.org/blog/2021/04/find-test.html

                                                                                                          More:

                                                                                                          https://github.com/oilshell/oil/wiki/Shell-WTFs

                                                                                                          https://github.com/oilshell/oil/wiki/Language-Design-Principles

                                                                                                  1. 12
                                                                                                    type Ruleset []Rule
                                                                                                    
                                                                                                    func (r Ruleset) Match(path string) (*Rule, error) 
                                                                                                    

                                                                                                    I understand why the author says they can’t change Ruleset to []*Rule or Match to func(string) (Rule, error), but in my experience, taking a pointer *T from a slice []T is pretty much always an “optimization” I end up regretting because it inevitably gets broken when you innocently append to the slice later. “Oh, I’m not mutating the interior of the slice, just adding onto the end, it should be fine.” Nope, it will not be fine. So many subtle bugs that way. Either make the slice a slice of pointers or return a value copy, but don’t take pointer to a slice of values because you will get bugs later.

                                                                                                    1. 3

                                                                                                      Yeah returning a reference to an element if a mutable slice doesn’t feel great given the slice can be reallocated. That said I think that if the slice is reallocated the reference that’s returned will keep the GC from cleaning up the old block of memory. So while you do hang on to some extra memory, at least there isn’t a true dangling pointer causing a memory safety issue. Does that match your understanding?

                                                                                                      1. 5

                                                                                                        That’s correct, but it means that things work most of the time until they don’t. It’s the worst kind of bug because it only happens sometimes that you miss updates or unexpectedly overwrite an existing element.

                                                                                                        1. 2

                                                                                                          I see what you’re saying—the issue isn’t a dangling pointer, it’s that mutations through the pointer will only be reflected in the slice if the slice hasn’t been reallocated. Yeah that’d be a super subtle bug to track down, thanks for pointing that out.

                                                                                                      2. 2

                                                                                                        I don’t quite know why, but I very rarely mutate arrays (including appends) in the first place–or rather, I’ll build some slice by doing a bunch of appends (although I prefer to do a make([]T, size) upfront if possible) but by the time someone else has a reference to the slice, it’s functionally immutable so I don’t need to worry about this sort of append scenario.

                                                                                                        Ideally Go would have enforced immutability semantics so we could express that a slice can’t be changed and it’s safe to take references to its items, but programming in this style seems like the next best thing (returning copies is good, but copies can contain pointers to other things, so leaning hard on copies for safety can be a different kind of pain).

                                                                                                        1. 2

                                                                                                          What if you need to sort the slice? Or filter in place? That’s been a pain point for me because they seem like they should work, but you just messed up any pointers you already returned.

                                                                                                          1. 3

                                                                                                            I think I concur with weberc2, I treat any slice that crosses a callstack boundary as essentially immutable, e.g.

                                                                                                            • Any code that receives a slice treats it as immutable, if e.g. I want to make changes to a received slice, I allocate a new slice, fill it by walking the input slice, and operate on the fresh value

                                                                                                            • Any code that returns a slice should return a unique value, if e.g. a method on a struct wants to return what is ultimately a field of that struct which is a slice, I always make a new slice in the method, copy elements over, and return the fresh value

                                                                                                            Sorting a slice is something that I’d do prior to returning that slice. If a caller receives a slice, they’re free to do whatever to it, including appending and sorting, until they return it themselves. Filtering always looks like e.g.

                                                                                                            orig := f()
                                                                                                            
                                                                                                            var filtered []int                      // by default, or...
                                                                                                            //filtered := make([]int, 0, len(orig)) // ...if profiling supports it
                                                                                                            for _, v := range orig {
                                                                                                                if filterPass(v) { filtered = append(filtered, v) }
                                                                                                            }
                                                                                                            
                                                                                                            1. 1

                                                                                                              Usually they’re sorted before I hand out pointers. I have a static site generator that does this—I load up all of my posts into a slice, sort by date, and then pass out pointers for further processing. I’m not sure when I would sort or append after I had already passed out pointers.

                                                                                                          2. 2

                                                                                                            💯

                                                                                                            A value method receiver like (r Ruleset) expresses that the method doesn’t mutate the receiver. That expectation is violated if the receiver is a reference type (e.g. a slice) and the method returns a pointer to an interior member of that receiver (e.g. a pointer to an element of that slice).

                                                                                                            Things get even trickier when you consider callers who might call stuff that defines new slice 3-tuples which happen to re-use (part of) the backing array of that original slice. Here be dragons!

                                                                                                          1. 17
                                                                                                            • Self-hosted CI/CD system which is not Jenkins
                                                                                                            • Self-hosted dashboard system which is not Grafana
                                                                                                            • Self-hosted non-distributed tracing system
                                                                                                            1. 4

                                                                                                              Drone for your first one? Bob?

                                                                                                              1. 3

                                                                                                                I should clarify that by “self-hosted” I mean “by organizations employing teams of engineers” and not “by individuals”.

                                                                                                                1. 3
                                                                                                                  • Self-hosted email for normies.
                                                                                                                  • Self-hosted blog for normies.
                                                                                                                  • Self-hosted vlog for normies.
                                                                                                                  • Self-hosted distributed backup for the above… for normies of course.

                                                                                                                  While easy to use software is possible, I reckon making it is quite hard.

                                                                                                                  1. 2

                                                                                                                    Netdata?

                                                                                                                    1. 2
                                                                                                                      • Drone or woodpecker for CI?
                                                                                                                      • Kibana instead of grafana?
                                                                                                                      1. 1

                                                                                                                        At #PreviousJob, we used BuildKite, and it was fantastic - you could deploy the agent pretty much anywhere you wanted, and I found the configuration much easier than any other system I’ve used (such as GitHub Actions).

                                                                                                                        1. 1

                                                                                                                          I recently realized that what I really want out of a CI system is running locally. Ideally, with decoupled backends, so, it can run tasks in containers, but also just processes, for simpler projects.

                                                                                                                          Most CI configuration languages don’t really let you do that, so you need to duplicate the commands for building, linting, testing, etc.

                                                                                                                          There’s that terra something tool, but it requires containers always, I think.

                                                                                                                          1. 1

                                                                                                                            I had a couple (very) rough ideas on a CI system. One was to make the actual task configuration a Lua script with a bunch of predefined functions to make a declarative pipeline possible, but to also allow the user to drop into more imperative steps. Using Lua lets you more effectively sandbox the tasks than using something like the JVM, the runner could be much leaner, and users could possibly get real autocompletion for their steps instead of only relying on docs for some made up yaml DSL. I also really want to integrate it more deeply with metrics, so you can see annotations in Grafana when you deployed and have automatic rollback when something goes wrong.

                                                                                                                            1. 2

                                                                                                                              (nods) Though something like this remains, to me, the most ideal architecture.

                                                                                                                          1. 1

                                                                                                                            Linked lists are typically the first data structure introduced to computer science students, invariant of domain or language or whatever else. It’s at least a little funny to see them challenged on the basis of usefulness or safety or etc.

                                                                                                                            1. 3

                                                                                                                              It’s funny, but also worth doing. Linked lists might have been competitive on 80s hardware, but they’re extremely rarely the best tool for the job on modern hardware.

                                                                                                                              1. 2

                                                                                                                                Linked lists are abstract data structures, same as hash maps or binary trees or whatever else. Whether or not they’re the best tool for the job isn’t a function of the hardware on which they run, it’s a function of the problem they’re used to solve, right?

                                                                                                                                1. 3

                                                                                                                                  No, because constants matter and hardware changes constants. In particular, cost of arithmetics to memory load and store changed considerably through history of hardware, to detriment of linked lists.

                                                                                                                            1. 2

                                                                                                                              I’ve been vocal about Markdown being bad for the many of the reasons listed. The latest cramp about it is that there’s no defined way to do metadata. Want to include an author, editor, license, keywords, etc. well, tough luck. With AsciiDoc, Org Mode, reStructuredText, the specs define how to embed metadata in their documents. When I edit an ODF in LibreOffice or an SVG in Inkscape, there is a document properties menu for this. RAW images in Darktable, there’s a prominent “metadata editor” panel in the lighttable view. Having a way to embed metadata is the default behavior for digital creative work file formats, but Markdown doesn’t have it. There’s “front matter”, which is literally just an ill-defined, nonstandardized, non-portable table–and it took your document from being able to be rendered in many contexts to now looking like garbage on your Git forge with a table that you have to parse and realize isn’t the content (and this issue doesn’t exist with examples like AsciiDoc because the renderer knows in this context, it doesn’t need to render the metadata). Without metadata, you have attribution and licensing issues, and need to invent ad-hoc formats.

                                                                                                                              It contains 95% of the functionality that most people need to create a document.

                                                                                                                              This I 95% don’t agree with. Where Markdown really doesn’t shine: writing blog posts and documentation–a place where you need more robust features like metadata, admonitions, attributions for blockquotes, collapsible block, etc. Markdown cannot deliver without getting locked into a “flavor”. I’m still slowly in the process of redoing my personal blog and there’s a mess to unravel because I originally tied myself to both the limiting Markdown but also got locked into a static site generator with a specific format I had to follow. Luckily, developers can, right now, go out and choose a better format for their projects (and no, this won’t make contributions harder because the text is still text).

                                                                                                                              1. 3

                                                                                                                                The latest cramp about it is that there’s no defined way to do metadata. Want to include an author, editor, license, keywords, etc. well, tough luck . . . [if] you need more robust features like metadata, admonitions, attributions for blockquotes, collapsible block, etc. Markdown cannot deliver without getting locked into a “flavor”.

                                                                                                                                But Markdown isn’t meant to provide these capabilities! If you need these things, then Markdown isn’t a solution for your problem. That’s fine! It doesn’t mean Markdown needs to change. It means you should pick something else, I guess, right?

                                                                                                                                1. 2

                                                                                                                                  That’s true. It works for things like comments here on Lobste.rs. However, developers not exposed to alternatives, aren’t realizing when they should be switching tools for the job and invent a bunch of incompatible forks and tools rather than moving to the tool for the job. That said, with AsciiDoc, reStructuredText, Org Mode, etc. you can write simple stuff too. It seems we’ve set unrealistic expectations to many that if it’s good enough for a limited functionality required for a comments section, it’s good enough for my blog, my README, etc. and generally it isn’t.

                                                                                                                                  1. 2

                                                                                                                                    shrug I’m not sure that the capabilities afforded by markup languages more powerful than Markdown outweigh their costs, for the use cases that Markdown targets.

                                                                                                                                    As an analogy, consider technical diagrams. I think there are basically 2 types of diagrams worth using: boxes-and-lines diagrams, to express component relationships in a system; and sequence diagrams, to express information flow between components over time.

                                                                                                                                    I think raw ASCII is the best way to draw both of these diagrams, because the constraints that raw ASCII imposes on author expressiveness is actually virtuous. Anything that’s too complex to express in raw ASCII is probably too complex for typical consumers. Said another way, if you can’t draw it effectively in raw ASCII, then the thing you’re trying to draw probably needs to be simplified — broken up into multiple drawings, etc.

                                                                                                                                    I think the same thing is more or less true for anything you could conceivably write in Markdown. If you feel like Markdown is a limiting factor, and you need your README or blog post to express things that aren’t possible to express, then I think it’s usually — not always! but almost always — an indication that you’re trying to express more detail than is appropriate for your audience.

                                                                                                                                    Concretely: a README should probably not include semantic author attributions of its blockquotes. The consumers of READMEs do not benefit from this information, and the maintainers of READMEs suffer from the additional syntax required to express that concept. The same thing is more or less true for licensing information, keyword metadata, etc. etc.

                                                                                                                                2. 2

                                                                                                                                  writing blog posts

                                                                                                                                  I don’t know what goes into your blog posts but for mine, MD would be fine and I wish my current blogging site supported it for posts. It does for comments, which is very handy.

                                                                                                                                  1. 1

                                                                                                                                    Yeah. I have collapsible blocks. I use admonitions (and not breaking the semantics of the <blockquote> element). I attribute quotes properly. I have definition lists, glossaries, footnotes. My image tags have heights, widths, and titles. Sometimes my blocks and lists have titles that aren’t <h#> headings. Occassionally I need to add a one-off CSS class too. My blog has keywords, and descriptions, an author, licensing, and other metadata. When I use Asciidoctor, all of these features are built into the syntax and don’t require an ad hoc systems or additional tools (though I am electing to use Soupault to help iron the data into a format I like better). And without much thought I could turn these documents into a book in PDF form and not be littered with HTML tags. I can also host look at it on Gitea or GitLab if I needed for whatever reason and I don’t have eyesore elements to deal with my limitations.

                                                                                                                                    I would venture that your blog needs some of these features too, but you’re probably relying on an ad hoc format like front matter that prevents its portability, or are using some elements incorrectly (like list items with BOLD text instead of definition lists, or have inline HTML which ruins the plaintext aesthetic and breaks in some contexts.

                                                                                                                                    1. 3

                                                                                                                                      I would venture that your blog needs some of these features too

                                                                                                                                      Chuckle Nope.

                                                                                                                                      you’re probably relying on an ad hoc format like front matter

                                                                                                                                      If I were ¼ century younger, I’d reply LOLnope or something.

                                                                                                                                      Sometimes I remember to replace * bold * with actual bold tags, but that’s as close to formatting as it gets. No. It’s a quick place to put random thoughts, and in the instance of my tech blog, I generally copy-paste longer comments there to preserve them and offer them to a wider audience. Posts fairly regularly get shared on Lobste.rs so I guess it’s working.

                                                                                                                                      This is sort of the point I was getting at: Markdown does all I need and more, and criticism of it is generally from people who need a different tool and so are misaddressing their complaints.

                                                                                                                                      I am not in any way denigrating your blog or anything! I am just saying that there is a good place for quick’n’dirty tools and I like them.

                                                                                                                                      This is why I dislike “docs as code”: because programmers’ editors are awful tools for a writer and I really dislike using them. I don’t want or need syntax highlighting, or line moves, or pattern matching. I really do need a live word count though, and nothing give me that since MS Word for macOS.

                                                                                                                                      ADoc has uses. I am very happy that I do not need a single thing it does that MD doesn’t any more, and I wish more people used ADoc instead of MD.

                                                                                                                                      1. 2

                                                                                                                                        you’re probably relying on an ad hoc format like front matter that prevents its portability

                                                                                                                                        That’s right. YAML front matter. Looks good & works great.

                                                                                                                                        or have inline HTML which ruins the plaintext aesthetic and breaks in some contexts.

                                                                                                                                        Some. But, it’s easy to remember and looks a lot better than AsciiDoctor syntax 💁🏻‍♀️

                                                                                                                                        That’s the entire idea behind markdown: it doesn’t try to be the be-all, end-all. It’s a simple filter for when you wanna write HTML but don’t wanna have to type out every single br or p or a bref or h1. It’s an amazing tool on the editing side and that’s all it needs to be.

                                                                                                                                        1. 2

                                                                                                                                          If you want to blog like you’re writing a thesis, more power to you. I personally just want to put pixels on screens, without having to worry about not closing tags. Markdown fills that need admirably.

                                                                                                                                          1. 1

                                                                                                                                            Sure, but a lot of people don’t pay attention to the generated HTML and end up with unsemantic elements that ruin other tools trying to consume the HTML. In cases like excluding width/height on images lead to performance and rendering issues. A lot of this is exacerbated by the lack of robustness and because of folks without blame not wanting to “worry about not closing tags”, use the wrong semantics due to the tools afforded by Markdown. If just getting text out is the goal and not following the HTML semantics, then Gemini seems like the format to choose.

                                                                                                                                            A specific example is admonitions where you see some folks using **NOTE:** foo, but this leads to just a plain paragraph with bold text with nothing to separate it from the main content (and some don’t all caps it, and other don’t bold it). Other implementation overload and abuse the > for block quotes (with broken implementations like GitHub’s still using the <blockquote> when generated which isn’t semantic and violates the spec of “a section that is quoted from another source” instead of using something semantic like <aside> or generic like <div>) and the label part isn’t agreed on in these overloaded block quote ideas. Microsoft does something with [!note].The CommonMark RFC is for blocks starting like :::note, but it seems most aren’t using for one reason or the other. Admonitions are very common for code documentation, and AsciiDoc and reStructuredText don’t have competing ways to write it–it’s a part of the spec and doesn’t break semantics.

                                                                                                                                            1. 2

                                                                                                                                              I use <aside> just fine on my blog, I use it all the time. I use it by simply typing <aside> which is easy enough to remember without having to do anything special. This is why dropping down to HTML is so awesome and part of Markdown’s magic.

                                                                                                                                              If just getting text out is the goal and not following the HTML semantics, then Gemini seems like the format to choose.

                                                                                                                                              Yes. It works. Just like plain text email works or plain text SMS text messages work.

                                                                                                                                              1. 1

                                                                                                                                                The easy of adding inline HTML isn’t the issue, it’s the use of unsemantic elements. This may be unintentional–especially for folks that haven’t read chunks of the HTML spec–but it’s still causing the wrong elements to be used. An informational admonition is complementary to the content, not a direct part of the document (which is why it’s often rendered in some form of call out). The blockquotes-for-admonition example mentioned is straight up wrong rather than an missing wrapper though. And when you see people create block quotes like

                                                                                                                                                Arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don’t care about free speech because you have nothing to say.

                                                                                                                                                ― Edward Snowden, https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/

                                                                                                                                                > Arguing that you don't care about the right to privacy because you have
                                                                                                                                                > nothing to hide is no different than saying you don't care about free speech
                                                                                                                                                > because you have nothing to say.
                                                                                                                                                
                                                                                                                                                ― Edward Snowden, https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/
                                                                                                                                                

                                                                                                                                                Now the quote attribution is in a generic <p> paragraph tag and not properly grouped with <blockquote>

                                                                                                                                                Arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don’t care about free speech because you have nothing to say.

                                                                                                                                                ― Edward Snowden, https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/

                                                                                                                                                > Arguing that you don't care about the right to privacy because you have 
                                                                                                                                                > nothing to hide is no different than saying you don't care about free speech 
                                                                                                                                                > because you have nothing to say.
                                                                                                                                                >
                                                                                                                                                > ― Edward Snowden, https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/ 
                                                                                                                                                

                                                                                                                                                This isn’t correct either because you’re missing the <cite>. So does this mean are you going to always going to bother writing for every attributable quote (assuming this is the structure you want; I know <figure> is common too)? Will you remember to use the correct structure every time?

                                                                                                                                                <div class="Quote">
                                                                                                                                                > Arguing that you don't care about the right to privacy because you have
                                                                                                                                                > nothing to hide is no different than saying you don't care about free speech
                                                                                                                                                > because you have nothing to say.
                                                                                                                                                >
                                                                                                                                                > <footer class="Quote-attribution">― Edward Snowden, <cite>https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/</cite></div></footer>
                                                                                                                                                </div>
                                                                                                                                                

                                                                                                                                                when in AsciiDoc you can do this and not think about the HTML semantics as they are handled for you

                                                                                                                                                [quote, Edward Snowden, https://www.reddit.com/r/IAmA/comments/36ru89/just_days_left_to_kill_mass_surveillance_under/crglgh2/]
                                                                                                                                                Arguing that you don't care about the right to privacy because you have
                                                                                                                                                nothing to hide is no different than saying you don't care about free speech
                                                                                                                                                because you have nothing to say.
                                                                                                                                                
                                                                                                                                                1. 3

                                                                                                                                                  This is exactly the level of programmer-brain semantic nitpicking that 99% of the population never should have to learn.

                                                                                                                                                  This is why in the olden days typographer was a separate job from journalist or authors, and these days why template designer is a separate job. So writers can be free to just write.

                                                                                                                                                  The fact that AsciiDoc has a way to do it is a strike against it for general users since they’re gonna be seen as dumb if they don’t do it semantically properly.

                                                                                                                                                  It’s possible to make a post-processor on the template language level to detect and fix those “footer”/“header” quote attributions by looking for foo bar: before the quote or –foo bar after the quote. (I should go set something like that up, thanks for reminding me; although in this case your proposed correct structure is incorrect since semantics can’t live on the class/id level and a footer is only appropriate for a document or section.) I don’t mind this sort of semantical type setting for the HTML target specifically. I do type <cite> and <i> all the time (and use Markdown’s shortcut for em. Common things super easy while rare things like i and cite in a normal, straightforward style. That is the way).

                                                                                                                                                  At the same time, I super appreciate that on gemtext, it’s not necessary. You just write and you’re done. Same goes for email, SMS, tweets, Mastodon etc. It’s just loosey-goosey text and it’s so functional and pragmatic.

                                                                                                                                                  Different formats have different semantical salience levels. On email you’ve got to get the “To:” correctly snd it’s also good to have “Subject:” and/or a “Body:”. In a printed book you might want page numbers.

                                                                                                                                                  The more of these semantical salience levels you slather onto a writer’s yoked shoulders, the worse their writing will be.

                                                                                                                                                  Also, the entire idea of the semantic web kinda failed. The more rules there are, the more rulebreaking texts there are. What’s the point? Some super web crawler evil AI gonna be even more correct in its dissection of humanity? Some of these semantic markups help screenreaders if they are applied consistently, like table navigation, but some don’t.

                                                                                                                                                  I don’t wanna have to learn AsciiDoc’s block structure syntax 💁🏻‍♀️ on top of having to learn the underlying HTML I also need to learn the semantics ins and out of (in order to be motivated to even use the AsciiDoc syntax in the first place)

                                                                                                                                                  1. 1

                                                                                                                                                    I disagree. For me the purpose of the lightweight markup language is to hide having to remember some of the difficult semantics in a portable syntax that can be exported to many formats without having to think too hard about the source material. The semantic web still give you a) good heuristics to follow (solving the “naming things” problem) and b) lets you skip needing to add role=* to elements to help with assistive technology. If you go further and use microdata, then you can free up needing to layout your markup in a very specific way for tools like the web crawlers–it also can disambiguates to a machine (i.e. is this date for created, updated, or published?) something that may be clear to a user visually reading content.

                                                                                                                                                    If you don’t want to think about it too much, then it’s okay to use <div> and <span>, but it’s not okay to use the wrong element (e.g. using a <blockquote> when it’s not quoting anything)

                                                                                                                                                    I agree that Gemtext is the place for that sort of thing. In many ways, folks should aim to use *.txt files if the goal isn’t to target HTML. GameFAQs was a better site when guides were plaintext and full of ASCII art over the HTML files–however, these formats don’t do the best job with accessibility or machines to understand the semantics. If SEO were less important, then still see curated lists like GameFAQs and Yahoo’s old search, but we have to appease the machines and if you play along with the semantic web, then there less ambiguous content for machines to error on. And at the end of the day, I find AsciiDoc’s plaintext semantics a lot easier to read than HTML as plaintext while being close enough to the ASCII text of the past.

                                                                                                                                                    1. 3

                                                                                                                                                      If you don’t want to think about it too much, then it’s okay to use and , but it’s not okay to use the wrong element (e.g. using a when it’s not quoting anything)

                                                                                                                                                      Right. I don’t use any divs personally because I have templating that does section, blockquote etc personally but I’ve argued again and again that Markdown-based languages, like on Reddit or here on lobste.rs, should have i instead of em since i is never wrong (and sometimes the only right one) but em can be wrong.

                                                                                                                                                      we have to appease the machines

                                                                                                                                                      Screenreaders yeah, search engine’s no, we don’t. If peeps wanna find the site they can bookmark it ♥ or use links.

                                                                                                                                                      a portable syntax that can be exported to many formats

                                                                                                                                                      To me, the lightweight format doesn’t itself have to be this über catch-all ultra portable thing. If a good one existed (which is maybe not even possible, as we learned from DocBook’s failures), we could then have a markdown-like language that dropped into it. Like, take DocBook again as an example. Writing it becomes much simpler with a li’l filter that does para and emphasis and other super common things, while the more nitty gritty things like the info block can live in a template written directly in the DocBook language.

                                                                                                                                                      Put another way: the PSTCBETMF needs to be very broad in order to cover all the features of all possible targets while the LYAWI (language you’re actually writing in) only needs to cover the most common, everyday features of one expression source.

                                                                                                                                                      Like, take my text overview page: https://idiomdrottning.org/texts
                                                                                                                                                      There’s no way I’d wanna write that in Markdown or AsciiDoc or like YAML or HAML or Textile or something. It has sections and columns and links and all kinds of stuff. (Instead it’s all generated from a template I wrote with Liquid and XHTML directly.) Markdown is only good for writing text but it’s damn good there.

                                                                                                                                                      the lightweight markup language is to hide having to remember some of the difficult semantics

                                                                                                                                                      You’ve identified exactly where we disagree there. It becomes the worst of all worlds:

                                                                                                                                                      • you have to know/understand the underlying semantics (just like you have to when you’re writing a template)
                                                                                                                                                      • you have to know/remember the way the lightweight markup language represents those semantics in all its exact fiddliness with brackets and commas and whitespace
                                                                                                                                                      • you have to the stare at all that syntax in all its ugliness even though it’s only like ten percent “lighter” than the target language
                                                                                                                                                      • you might have to flip back and forth to a render/preview to see if it worked

                                                                                                                                                      Take links for example:

                                                                                                                                                      https://wikipedia.org[Wikipedia] is an example in AsciiDoc.
                                                                                                                                                      
                                                                                                                                                      <a href="https://wikipedia.org/">Wikipedia</a> is an example when dropping down to HTML.
                                                                                                                                                      

                                                                                                                                                      The AsciiDoc version is barely any more readable! It totally breaks the reading flow.

                                                                                                                                                      While [Wikipedia] is how it looks in Markdown.
                                                                                                                                                      

                                                                                                                                                      Much more readable!

                                                                                                                                                      1. 1

                                                                                                                                                        I think we agree about our disagreement here other than how and where we feel that line should split (and I can see your point). I don’t mind learning and remembering a bit more syntax to have more features and more easily turn it a document into a PDF or slides. I do care about the semantics quite a bit though.


                                                                                                                                                        The only call out here is that the

                                                                                                                                                        https://wikipedia.org[Wikipedia]
                                                                                                                                                        

                                                                                                                                                        is closer to

                                                                                                                                                        <a href="https://wikipedia.org/">Wikipedia</a>
                                                                                                                                                        

                                                                                                                                                        in that it’s URL then text

                                                                                                                                                        and you can do

                                                                                                                                                        :wikipedia: link:https://wikipedia.org/
                                                                                                                                                        
                                                                                                                                                        {wikipedia}[Wikpedia]
                                                                                                                                                        

                                                                                                                                                        Just like you still need to label it in Markdown (but there’s a short hand if key matches your text)

                                                                                                                                                        [Wikipedia][wikipedia]
                                                                                                                                                        
                                                                                                                                                        [wikipedia]: https://wikipedia.org/
                                                                                                                                                        
                                                                                                                                                        1. 2

                                                                                                                                                          URL before text?!😱 Cart before horse!

                                                                                                                                                          Footnote-superscripts don’t come before the phrase they’re attached to.
                                                                                                                                                          The page you’re gonna visit next doesn’t come before the text you’re reading now.

                                                                                                                                                          I don’t mind learning and remembering a bit more syntax to have more features

                                                                                                                                                          Well… we all wants things to be as simple as possible. But then I say no simpler. If I want syntax for all those features, I know where to find it. I don’t need to create a semaphore system of smoke signals and shadow puppets where I can write what I in turn meant to actually write. There’s no reason the syntax for those special features should be like AsciiDoc’s mystical squares and commas instead of a normal, full-featured, real, explicit syntax.

                                                                                                                                                          It’s not as if I need to write those “explicit syntax” parts tediously, character by character, either; my blogging system isn’t a typewriter so I’ve set it up to prompt me for things. What category should this be? What is the URL to that thing I square bracketed?

                                                                                                                                                          And when I need to do weirder things like this treasure generator I have an escape hatch from the simple convenience layer (Markdown) into the real thing (HTML). Right there in the same blog.

                                                                                                                                                          1. 1

                                                                                                                                                            I learned recently that Asciidoctor supports Unicode bullets, , for list items, and that is enough to sell me. Type what you mean.

                                                                                                                                                            1. 2

                                                                                                                                                              Yeah, I’ve talked to Gruber about adding them to Markdown too.

                                                                                                                                                              It’s just that the square bracket percent sign comma magic in AsciiDoc is such an unsell.

                                                                                                                                                  2. 2

                                                                                                                                                    The easy of adding inline HTML isn’t the issue, it’s the use of unsemantic elements.

                                                                                                                                                    Authors who want to quote something will have a few things at hand:

                                                                                                                                                    1. The text itself (guaranteed)
                                                                                                                                                    2. The author (likely)
                                                                                                                                                    3. An origin URL (possible)

                                                                                                                                                    A usable markup syntax for quotes can’t assume that all of these bits of metadata will be available. If all you have is #1, you shouldn’t have to know about #2 or #3 to express your intent. And, in all cases, you should be able to express your intent without deep knowledge of the tool you’re using.

                                                                                                                                                    If you wanna quote something, the ASCII text default way to do that, understood by every human on the internet, is more or less this:

                                                                                                                                                    > Your quote here.
                                                                                                                                                    

                                                                                                                                                    Whatever syntax you have for quoting has got to support that as a base case. Given that, the question becomes how to optionally layer-on the other bits of metadata — author, and origin URL — in a way that works with this base assumption. And

                                                                                                                                                    [quote, Author Lastname, https://example.com/path/to/document]
                                                                                                                                                    Your quote here.
                                                                                                                                                    

                                                                                                                                                    is I guess not it?

                                                                                                                                                      1. 1

                                                                                                                                                        Sure, but the optionality is itself a problem. Even if I don’t use a particular feature myself, I still have to know about it, in order to understand the output of others.

                                                                                                                                                        Markdown suffers from this, too. You can maybe make a case for both the - and * markers for un-numbered lists, but I can’t see any reason for the === syntax to denote headings. There are probably other ambiguities, too.

                                                                                                                                                        1. 1

                                                                                                                                                          In the ASCII days it was common to use heading like (as is the case with reStructuredText)

                                                                                                                                                          Title
                                                                                                                                                          ====
                                                                                                                                                          

                                                                                                                                                          This is supported in AsciiDoc, but they probably found it a bit obtuse and impractical in many cases and used the = sigil. What on Earth would make # more meaningful?

                                                                                                                                                          1. 2

                                                                                                                                                            # has several structural advantages over ===, I think. It sits on the same line as the heading, so it’s easier to parse for both humans and computers. It supports arbitrary levels thru repetition, and it’s intuitive: # H1 and ## H2 and etc. make sense. And because it prefixes the header text rather than underlining it, the typography always basically “works”.

                                                                                                                                                            Your Heading
                                                                                                                                                            ============
                                                                                                                                                            Should be fully underlined, if you're going to underline it.
                                                                                                                                                            
                                                                                                                                                            Whereas
                                                                                                                                                            ===
                                                                                                                                                            If you don't it looks quite silly ;)
                                                                                                                                                            

                                                                                                                                                            c.f.

                                                                                                                                                            # Your heading
                                                                                                                                                            ## Whereas
                                                                                                                                                            ### Subsection with long name
                                                                                                                                                            

                                                                                                                                                            It’s just much cleaner. (shrug) all IMO of course

                                                                                                                                                            1. 1

                                                                                                                                                              Wait what? Did you think underline was the only style supported? AsciiDoc normally, from my experience, wants users to do

                                                                                                                                                              = Your heading
                                                                                                                                                              == Whereas
                                                                                                                                                              === Subsection with long name
                                                                                                                                                              

                                                                                                                                                              That is no different other than the sigil. You can even see this in practice in the raw version of the heading documentation for Asciidoctor, as well as the rendered docs. And due to “Section levels must be nested logically.”, you can’t nest improperly, something Markdown doesn’t enforce and was a pain I had to deal with recently.


                                                                                                                                                              The only time I personally use the underline option is for the main title for stylistic reasons, because it adds visual separation for the metdata and makes it stand out in the visual hierarchy as the document title

                                                                                                                                                              My Title: and its Subtitle
                                                                                                                                                              ==========================
                                                                                                                                                              toastal
                                                                                                                                                              :keywords: foo, bar, baz
                                                                                                                                                              :description: a mock post
                                                                                                                                                              :license-url: https://creativecommons.org/licenses/by-nc-sa/4.0/
                                                                                                                                                              :license-title: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) 
                                                                                                                                                              
                                                                                                                                                              [abstract]
                                                                                                                                                              --
                                                                                                                                                              Today we will discuss something
                                                                                                                                                              --
                                                                                                                                                              
                                                                                                                                                              == My first section
                                                                                                                                                              
                                                                                                                                                              And its paragraph
                                                                                                                                                              
                                                                                                                                                              === Subsection with long name
                                                                                                                                                              
                                                                                                                                                              Another paragraph
                                                                                                                                                              
                                                                                                                                                              1. 1

                                                                                                                                                                Wait what? Did you think underline was the only style supported? AsciiDoc normally, from my experience, wants users to do . . .

                                                                                                                                                                Yeah, I did! Neat that AsciiDoc lets you use = as a prefix like that. Markdown doesn’t, and that was my point of reference. I don’t use AsciiDoc, in large part due to the sort of optionality we’re talking about right now. If a language allows its users to express a given outcome (e.g. an tag) in more than one way, I see that as a problem that needs to be fixed. YMMV, of course!

                                                                                                                                                                nest improperly

                                                                                                                                                                What does “nesting improperly” mean? AFAIK this isn’t a concept that exists in the headers produced by e.g. # Heading or ## Subheading in Markdown. That syntax translates directly to <h1> or <h2> which are tags that need to be closed but otherwise carry no semantic value.

                                                                                                                                                                The only time I . . .

                                                                                                                                                                The text you wrote there is pretty complex in terms of sigils and syntax and demarcated sections and etc. For a human reader of that plain text, it is IMO substantially more difficult to understand than the XML or HTML that would be produced from it. And again IMO this is a direct result of the set of features, and the optionality of syntax, which AsciiDoc (?) provides. It’s fine if you don’t agree! People are different, no problem. But if you PR’d a project I maintained and added docs in that format, hard no from me 😉


                                                                                                                                                                edit: If all of that information is strictly necessary, and this Markdown version

                                                                                                                                                                # My Title: and its Subtitle
                                                                                                                                                                
                                                                                                                                                                ## Metadata
                                                                                                                                                                
                                                                                                                                                                - Keywords: foo, bar, baz
                                                                                                                                                                - Description: a mock post
                                                                                                                                                                - License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)
                                                                                                                                                                
                                                                                                                                                                ## Abstract 
                                                                                                                                                                
                                                                                                                                                                Today we will discuss something.
                                                                                                                                                                
                                                                                                                                                                ## My first section
                                                                                                                                                                
                                                                                                                                                                And its paragraph.
                                                                                                                                                                
                                                                                                                                                                ### Subsection with long name
                                                                                                                                                                
                                                                                                                                                                Another paragraph.
                                                                                                                                                                

                                                                                                                                                                is in some respect insufficient in terms of expressiveness, then I just can’t grok the value prop of an intermediary language between that and straight up HTML/XML. Those languages are meant to be basically human-readable, right?

                                                                                                                                                                1. 1

                                                                                                                                                                  one heading style

                                                                                                                                                                  This may be a legacy thing, as a lot of old docs supported the underline style. It makes parsing more complicated which is relevant, but as a user, it’s not a big deal and can be skipped.

                                                                                                                                                                  nesting headings improperly

                                                                                                                                                                  From their docs:

                                                                                                                                                                  Section levels must be nested logically. There are two rules you must follow:

                                                                                                                                                                  • A document can only have multiple level 0 sections if the doctype is set to book.
                                                                                                                                                                  • The first level 0 section is the document title; subsequent level 0 sections represent parts.
                                                                                                                                                                  • Section levels cannot be skipped when nesting sections (e.g., you can’t nest a level 5 section directly inside a level 3 section; an intermediary level 4 section is required).

                                                                                                                                                                  I have seen users do this wrong–especially when focusing on the visual style a heading provides instead of the semantic one for the document (e.g. Why did you make this an <h4>? Because I wanted the text smaller.).

                                                                                                                                                                  your markdown

                                                                                                                                                                  What’s happened here is that you’ve built an ad-hoc system. It’s not that it’s not readable as plaintext–it’s easier to read–but it caries no underlying semantic value.

                                                                                                                                                                  • Metadata and abstract as regular headings

                                                                                                                                                                    How is the parse reasonable supposed to assume these are not just sections with content? How would I create a regular sections called “Abstract” or “Metadata”?

                                                                                                                                                                  • Metadata as a list

                                                                                                                                                                    Did you want a definition list? (Oh, Markdown doesn’t support this 🙃). These get rendered as plain <li> elements without any concept of key/value that. Do I have to escape : if I need it in a key? What’s up with keywords being comma-separated? Is this another ad-hoc convention (vs. <dl><dt>Keywords</dt><dd>foo</dd><dd>bar</dd><dd>baz</dd></dl>)?

                                                                                                                                                                    In the case of Asciidoctor, I can do something with this metadata labeled as such:

                                                                                                                                                                    • my metadata can be correctly tagged in HTML, PDF, DocBook, and other outputs in their expected formats
                                                                                                                                                                    • metadata can be reused inside the document since it’s the same as the ‘variable’ syntax
                                                                                                                                                                    • I can use docfile formats to add additional metadata to various parts of files if I need something special
                                                                                                                                                                    • other ‘consumers’ of my file can happily ignore them (e.g. Codeberg doesn’t need to display the license of that document or the author, etc. (or could put them in the <head> but not rendered in the document)–with Markdown, because it’s ad hoc, they can only assume it always needs to be displayed)
                                                                                                                                                                  • Abstract as a generic section

                                                                                                                                                                    Abstracts are a special part of the document with a special meaning. You often want to style it differently. This was one of this issues I had recently trying to parse a lot of different people’s Markdown files they sent me; some people had an Abstract with a heading, some had a paragraph with nothing to delimit it and others started with a generic paragraph that wasn’t an abstract. Others started with a backstory instead. Without a heading, you can’t reasonably assume a block is the abstract or not or what order these elements are in. If the user specifies it, it’s clear.

                                                                                                                                                                  1. 1

                                                                                                                                                                    What’s happened here is that you’ve built an ad-hoc system. It’s not that it’s not readable as plaintext–it’s easier to read–but it caries no underlying semantic value.

                                                                                                                                                                    So I guess a good way of summarizing what I’m trying to say here is that if you need to express “underlying semantic value” in a document, you should not use a plain-text markup language like Markdown (for sure) or AsciiDoc (IMO) in the first place. These tools are not meant to provide a high degree of semantic expressiveness. They’re meant to provide the bare minimum set of capabilities — mostly typographical — while maintaining plain-text legibility.

                                                                                                                                                                    How is the [parser] reasonably supposed to . . .

                                                                                                                                                                    Ah! Maybe this is a point. Tools like Markdown aren’t meant to produce mechanically parseable output. They’re meant to produce typographically acceptable documents that can be “parsed” by human readers.

                                                                                                                                                                    Maybe AsciiDoc tries to solve a different set of use cases. That’s cool! But it means the tools aren’t really comparable. When I’m writing a GitHub comment or whatever, semantic annotation features are actively harmful to my user experience.

                                                                                                                                                                    If you want semantically meaningful, mechanically parsable documents, then why not just write HTML/XML directly? With care, it’s absolutely human-readable — it was designed to be so! And you don’t lose any expressive power in translation.

                                                                                                                                                                    1. 1

                                                                                                                                                                      I wouldn’t disagree with the assessment, but I would disagree that human-readable isn’t the only goal. Machines parse our markup for SEO, for web scraping, and we can use that metadata in interesting ways like creating an outline or feed. In these cases, having a semantic makes these transformations easier and parsing more accessible. I feel the advantages of AsciiDoc or reStructuredText are big enough and don’t change enough that they’re still “lightweight markup languages” while having more expressiveness. While I had an hunch this mattered and switched on personal stuff a while ago, but it was much more apparent whenl I touched a recent project in which I was tasked to wrangle multiple authors’ different Markdown style that it became so apparent that the project really could have used a switch in the syntax for compliance and consistency. We ended up having to build sidecar files for the metadata but authors chosen a format with metadata, this could have been avoided, easier to manage, and less buggy.

                                                                                                                                                    1. 1

                                                                                                                                                      Now the quote attribution is in a generic <p> paragraph tag and not properly grouped with <blockquote>

                                                                                                                                                      In this document the attribution is in a <p> paragraph outside the which implies it is correct according to the standard:

                                                                                                                                                      https://html.spec.whatwg.org/multipage/grouping-content.html#the-blockquote-element

                                                                                                                                                      (second example, author is “Stephen Roberts”)

                                                                                                                                                      Edit re: the cite attribute, the same document states

                                                                                                                                                      If the cite attribute is present, it must be a valid URL potentially surrounded by spaces. […] User agents may allow users to follow such citation links, but they are primarily intended for private use (e.g., by server-side scripts collecting statistics about a site’s use of quotations), not for readers.

                                                                                                                                                      (My emphasis)

                                                                                                                                                      I’d much rather have the citation source linked to the attribution “element”, like so, to make it easier for readers to follow the link. I don’t really care about statistics about a site’s use of quotations.

                                                                                                                                                      1. 2

                                                                                                                                                        You are correct!

                                                                                                                                                        I had thought the attribution needed to be outside the <blockquote>, but this isn’t the default layout for Asciidoctor and I was assuming they were right in their semantics. I doubled back on what I originally written under those assumptions, but it seems they are wrong and I’m happy I had another opportunity to reread the spec (even if it was just a few lines down from what I had linked for how <blockquote> was being misused for admontions). I will raise an issue with them now. I think they know about it (there is a migration to adopt more HTML5 and semantic HTML in the future) and the asciidoctor-html5s does it closely enough.

                                                                                                                                                        1. 1

                                                                                                                                                          Right, it should’ve been

                                                                                                                                                          <blockquote cite="url to reddit">text of quote</blockquote>
                                                                                                                                                          
                                                                                                                                            2. 1

                                                                                                                                              What’s caught my eye recently is the asciidoctor-html5s alternative backend for Asciidoctor. I know there are long-term goals for the Asciidoc project to modernize its HTML representation, but this is something that can be used now.

                                                                                                                                              My biggest missing feature for AsciiDoc would be:

                                                                                                                                              • a shorthand for <abbr> tags for accessibility–ideally in a manner where the <abbr>s would be automatically added as I wrote and they were automatically appended to the glossary. You don’t realize how much jargon people write until you step a little out of your own space
                                                                                                                                              • more metadata options: editors, collaborators, translators (not just authors)
                                                                                                                                              1. 3

                                                                                                                                                I added support for <abbr> tags to my own markup language. I define the bunch that I use at the top of the document and anywhere the acronym or abbreviation appear in the text, an appropriate <abbr> tag is generated. It fails if you use an acronym with two different meanings (e.g. “the IRA (Irish Republican Army) has its own IRA (Individual Retirement Account)” for a silly example) but it has rarely come up. I have had to special case a bunch since acronyms can come in many forms (D&D, GHz, IPv4, NaNoGenMo) but it generally works.

                                                                                                                                                1. 1

                                                                                                                                                  I use <abbr title="Three Letter Acronym">TLA</abbr>.

                                                                                                                                                  Perfect example of why just dropping down to the HTML you already know is better than having to figure out some arcane syntax to learn in order to generate the HTML semantics that you also need to know & be aware of in order to use them.

                                                                                                                                                  1. 1

                                                                                                                                                    What happens when you want to convert this to another format that isn’t HTML? This is lock-in–lock-in you might be okay with, but it does hamper other uses.

                                                                                                                                                  2. 1

                                                                                                                                                    I just have a bunch in a text file and I wrote a Blosxom plugin to replace them when the HTML is generated.

                                                                                                                                                    Here’s my announcement https://gerikson.com/blog/scrivener/Defining-acronyms.html, old enough to vote next year….

                                                                                                                                                    1. 1

                                                                                                                                                      404 × 2

                                                                                                                                                      1. 1

                                                                                                                                                        Whoops, private repo. Updated the link.

                                                                                                                                                  3. 1

                                                                                                                                                    More builtin* metadata options (docinfo files do exist)

                                                                                                                                                1. 6

                                                                                                                                                  I think the main reason why markdown gets hate is that people don’t understand what it is. It’s a way to write what people wrote before markdown, in emails, on newsgroups, in text files. It’s a formatting that is natural if you do it that way.

                                                                                                                                                  But when you then end up looking at markdown files where headings don’t look like headings, lists not like lists and especially tables not like tables, because people only look at the output of their editor plugin, etc. it’s also clear that they actually want to write in something else.

                                                                                                                                                  The main criticisms of Markdown seem to all match with its original use case. The problem is that people feel or are forced to use it, for situations where they at least think something else would be better. With GitHub making Markdown popular for README files, and these nowadays becoming pretty much the equivalent of static HTML pages, which in and of itself is at least a questionable direction.

                                                                                                                                                  Markdown isn’t that clear cut markup language spec, because it’s text to be read, that is just defined enough for computers to render out that text into HTML.

                                                                                                                                                  I see parallels there with YAML, which also found its way to be config files to replace the equally bad choice of JSON to be config files. YAML was also mostly about readable adaptable text, and not something that would really have to deal with types a lot. Now we use it for giant config files, instead of stuff that was actually made for that purpose, like UCL, toml, and the only ones realizing that seem to be the Hashicorp folks with HCL.

                                                                                                                                                  The reason I bring this up, is because it shows a pattern where something is great, people use it, like to use it, but then more and more people try to squeeze it into other realms, because yeah, it’s possible, but it might not be a good idea. And when it end up being used everywhere instead of only where it makes sense, people write articles like this about how they suck.

                                                                                                                                                  There is this habit of thinking that you should only use one way, probably the most commonly used way of achieving something, but when it comes at the cost of having to bend and twist things to make it work or when simply it’s not the right tool for the job, just go and use something else.

                                                                                                                                                  I also think this often goes into the whole “modern”, “best practice”, “commonly used” area. Just because any markup language is commonly used, it doesn’t mean you should use it to do everything with it. It’s not like you use only one knife from spreading butter, to sawing boards, just because you can and already own it or know how to use it.

                                                                                                                                                  Also sometimes it just takes some experimentation in early development, to figure out what works and does feel right, and what you might want to replace.

                                                                                                                                                  Please don’t act like there can only be Markdown or HTML. People don’t usually come up with new markup languages, because they hope it will abolish all the other ones, and if they do I would be skeptical and stay away from it, because it’s almost bound to soon be “that weird old format”.

                                                                                                                                                  And yes, there’s good reasons why these problems arise. A lot of developers rightfully want to standardize how to do things, and want to have things flexible and generic, but following it too much leads to weird stuff, like Markdown extensions, programming languages with a bazillion keywords, and every project looking like it uses a totally different language, because every language feature ever invented in any programming language gets added, or functions, methods and APIs accidentally becoming their own Turing complete language, because of all the parameters you can pass.

                                                                                                                                                  With compatibility being such a hyped term, I think it would be could for the industry if we’d use the right, rather than the most popular tools and building blocks to compose our systems. I also think it would help if not every new tool that came along would call every other “obsolete” or “legacy”, just because it goes a different route. People will take that to heart and think they have to shove the new tool into every little project. Not every new standard and technology must always take over and replace everything else, it can even remotely be imagined to replace. Doctors with their MRIs and what not also still use their eyes, a lamp and a wooden stick to take a look into your throat.

                                                                                                                                                  1. 3

                                                                                                                                                    +100.

                                                                                                                                                    Markdown is valuable because of its limitations, not in spite of them. Learn a handful of syntactical rules — header syntax, emphasis syntax, list syntax, link syntax, and rules about indentation — and you basically understand the entire thing. That’s the whole ball game.

                                                                                                                                                  1. 6

                                                                                                                                                    This is, incidentally, a thing I dislike a lot about Rust stylistically. A lot of Rust is written chaining a bunch of map()s and or_else()s and such.

                                                                                                                                                    It’s passable if, even as in this article, it’s a straight transform. But it rarely is. There’s often side effects (either unintentional bad ones or good ones that’d make life simpler (eg, a max/min, without using tuples on tuples))… or implicit awaiting in async version of this chaining (how many things are happening in parallel? Hope you did the right buffering call!) and… it’s just a beast to debug. A for loop would be way more obvious in a lot of cases (if for no other reason than to crack open gdb)

                                                                                                                                                    1. 6

                                                                                                                                                      In my experience there’s a lot of cases when writing Rust code where you have a simple transform and do want to write it using chained maps and or_else (or even more concisely with the ? operator). When what you’re doing isn’t actually a simple transform, it’s definitely useful to resort to a non-functional construct like a for-loop, but that’s ideally the heavier and less common thing to do.

                                                                                                                                                      1. 2

                                                                                                                                                        Why would a for loop be “heavier” than a chain of transforms?

                                                                                                                                                        If anything, the for loop is easier to optimize by the compiler, and easier to understand by the layperson, right?

                                                                                                                                                        1. 1

                                                                                                                                                          I have no idea whether iterator chains or a for-loop is easier to optimize by the compiler - I’ve seen deep-dives into rustc compiler internals that have argued that iterator chains are actually faster at least sometimes, and I think the broader problem is that it’s difficult for a programmer to actually know which of several semantically-equivalent ways of writing a program will actually result in the most performant code.

                                                                                                                                                        2. 1

                                                                                                                                                          This lines up with my Java intuition as well, although there are so many different Java styles that I don’t claim it’s a universal thing other Java programmers would agree with. If something is doing explicit for loops to transform a collection into another collection, my assumption is either: 1) it’s legacy pre-Java-8 code, written before java.util.stream existed, or 2) it’s doing complex or wonky enough logic that it doesn’t map nicely onto the standard operations, so really does need to drop down to a custom loop.

                                                                                                                                                        3. 4

                                                                                                                                                          A lot of Rust is written chaining a bunch of map()s and or_else()s and such.

                                                                                                                                                          I used to do this a lot. The APIs are there. It’s so tempting when there’s a function like map_or_else that looks like it was designed to solve your exact problem. But I think it’s a trap, and as you start writing it often becomes more complicated than anticipated.

                                                                                                                                                          These days, I am more skeptical of the ‘functional’ style in Rust. I rely more on language features like match, ?, traits like From/TryFrom, and libraries like anyhow or serde. I couldn’t tell you why, but this is how I feel after using Rust for a couple years.

                                                                                                                                                          1. 3

                                                                                                                                                            I agree. Chaining in a single expression is usually less readable than a sequence of independent expressions.

                                                                                                                                                            1. 2

                                                                                                                                                              Yeah, we write a lot of functional heavily chained code in D, but it’s viable (IMO) because all our value types are immutable. No state, pure functions only. There’s a few keywords that are allowed to do mutation (each, sort), but we know to look out for them.

                                                                                                                                                            1. 19

                                                                                                                                                              I’m not an expert on linux, but as a developer who dabbles with it, systemd is awesome. Things feel much more consistent and discoverable. The services part alone, with cron and process monitoring features is great. No more need for init scripts, I can just stick a shell script into a service and a timer and call it a day.

                                                                                                                                                              1. 20

                                                                                                                                                                I’ve adminned Linux machines on a small scale (home/hobby, and a small web hosting company back in the days when servers were pets, not livestock) for over 20 years, and systemd has been absolutely a blessing to me in every way. Setting up new services is completely declarative now, and can include a lot of hardening that would double or triple the length of traditional init scripts. The documentation is not organized that well, but is remarkably complete and clear once you find what you’re looking for, whereas the documentation of traditional init scripts was a combination of oral history and reading the scripts to see what they did.

                                                                                                                                                                I’m not crazy about the tight coupling in systemd, nor its extension to places where it’s not needed (like home directories and DNS), but for service management, it’s 100% the right thing, and it puzzles me why Unix grognards hate it so much.

                                                                                                                                                                1. 1

                                                                                                                                                                  I can’t answer for all Unix grognards, just myself.

                                                                                                                                                                  It’s the tight coupling. And it’s the extension to take over everything. And it’s the binary format logging. And it’s the giganticness. And it’s the nonobviousness of the declarative config, where everything has a near-homonym that does something different. And it’s the history of the progenitor, whose previous projects were also widely adopted and broke things.

                                                                                                                                                                  What do you get if you remove all of the sources for those objections (besides the last one)?

                                                                                                                                                                  A small PID 1 init that launches a supervisor with well-defined interfaces that allow it to run your choice of daemons, with defaults for logging, start, stop, security, dependency… that can be overridden by a well-documented concise set of options. A project that values working well with others, reducing bugs by reducing code paths, and ease of debugging configs.

                                                                                                                                                                2. 5

                                                                                                                                                                  Single favorite feature of mine is systemd timers which get away from the arcane * 13 1 1 * <command> format and let you specify time like a human would eg. OnCalendar=weekly or OnCalendar=Sat,Sun 20:00. I do wish timers held or could hold their commands, but whatever I can read their frequency without needing a reference at least.

                                                                                                                                                                  1. 2

                                                                                                                                                                    I moved a scheduled job from an unsupported in-house version of cron to a systemd timer, it worked like a charm. I really loved the decoupling of the service definition (what to run) and the timer definition (when to run it).

                                                                                                                                                                    1. 1

                                                                                                                                                                      BSD systems use from to drive periodic, which lets you run a weekly shell script by just dropping it in the weekly directory. Most of the system-provided ones include rc.conf and run their job only if a specific enable flag is set.

                                                                                                                                                                    2. 1

                                                                                                                                                                      much more consistent

                                                                                                                                                                      systemctl status something.service
                                                                                                                                                                      
                                                                                                                                                                      journalctl -u something.service
                                                                                                                                                                      

                                                                                                                                                                      ???

                                                                                                                                                                      1. 5

                                                                                                                                                                        You do pretty much different things there. One command is showing status of service while another one is showing logs filtered to match particular unit. You can pass much more filters to journalctl and compose them, you cannot pass filters nor compose them to systemctl status.

                                                                                                                                                                        1. 2

                                                                                                                                                                          You’re describing the implementations of the commands as they exist. I understand that, and I understand why those differences exist. It doesn’t matter, though. If a system provides a set of binaries of the form <noun><verb> i.e. {journalctl,systemctl} then the clear user expectation is that they have the same semantics. If that’s not the case, then they should not have the same verb.

                                                                                                                                                                          If xctl only accepts a single argument (unit), but yctl acceps N units, then there exists a wide range of acceptable choices for how to specify arguments, absolutely. But allowing xctl unit and not xctl -u unit, while allowing yctl -u unit and not yctl unit? This definitely isn’t one of them.

                                                                                                                                                                    1. 14

                                                                                                                                                                      Slightly strange that a lot of this is about portability, but still prefers bash to POSIX sh. Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible, such as using [ or test rather than [[ - all of which are bash builtins.

                                                                                                                                                                      The TRACE thing seems unnecessarily more complicated than running sh -x script.

                                                                                                                                                                      1. 15

                                                                                                                                                                        Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible, such as using [ or test rather than [[ - all of which are bash builtins.

                                                                                                                                                                        I disagree—at least about your specific example. If you write shell scripts that specifically point to bash, then why not take advantages of the differences between [ and [[? (For example, word splitting: https://wiki.bash-hackers.org/syntax/ccmd/conditional_expression#word_splitting.)

                                                                                                                                                                        Note: this is a different question than “Should portable scripts prefer bash to POSIX shell?” My point is that scripts with bash in the shebang should absolutely use bash features.

                                                                                                                                                                        1. 5

                                                                                                                                                                          I get your point. My thoughts were that when I come across a bash script, I can still run it as a normal shell script like sh script and the fewer bashisms it contains, the easier it is for me to use (I don’t have bash installed).

                                                                                                                                                                          1. 2

                                                                                                                                                                            If you write shell scripts that specifically point to bash

                                                                                                                                                                            But that is the diametrical opposite of portability.

                                                                                                                                                                            1. 4

                                                                                                                                                                              But that is the diametrical opposite of portability.

                                                                                                                                                                              Right, but as I said, “Note: this is a different question than ‘Should portable scripts prefer bash to POSIX shell?’ My point is that scripts with bash in the shebang should absolutely use bash features.”

                                                                                                                                                                              The person I was replying to said “Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible.” I disagree with that, and I tried to make it clear that I wasn’t talking about portability per se.

                                                                                                                                                                              You can also see lollipopman’s comment for further arguments in favor of [[ once you are already using bash.

                                                                                                                                                                          2. 6

                                                                                                                                                                            “Portability” these days means “works on OS X and Linux”. Might as well get improvements from bash in that case.

                                                                                                                                                                            1. 4

                                                                                                                                                                              Which version of bash? Don’t know if they still do, but macOS used to ship ancient bash releases because of GPLv3

                                                                                                                                                                              1. 3

                                                                                                                                                                                “Portability” these days means “works on OS X and Linux”.

                                                                                                                                                                                To whom?

                                                                                                                                                                                1. 3

                                                                                                                                                                                  What percentage of people who would execute a script of the type described by this article are not Linux or macOS users, do you think?

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    I mean, I was being snarky, but that seems to be the case; other systems are routinely ignored, for better or worse.

                                                                                                                                                                                  2. 2

                                                                                                                                                                                    I’d argue portability these days means that it runs on many different systems.

                                                                                                                                                                                    1. 1

                                                                                                                                                                                      I wouldn’t call anything in bash an “improvement”…

                                                                                                                                                                                    2. 3

                                                                                                                                                                                      I find this idea that something “improves” by using a less capable shell nonsensical. What exactly improves except that you adhere to some standard nobody cares about. This purism buys you nothing. If you know that the target systems have bash, then use all its features.

                                                                                                                                                                                      1. 3

                                                                                                                                                                                        The first problem is that POSIX shell is an absolutely terrible scripting language, whereas bash is a merely awful one. If you reach the level of complexity where POSIX shell makes it hard to maintain then you are almost certainly at the point where you should ditch any kind of shell and use a programming language.

                                                                                                                                                                                        The second problem is the monoculture. For example, The shell shock vulnerability was so bad because everyone used bash for the scripts that were run from dhcpd and so a malicious user on the same broadcast domain could compromise any system that sent a dhcp request. This would have been far harder to exploit at scale if these scripts used /bin/sh and different systems used different implementations (mainstream systems use at least five different implementations of their default POSIX shell).

                                                                                                                                                                                        The third problem is performance. Bash is definitely not the fastest shell. FreeBSD uses the statically linked version of their POSIX shell for a bunch of things because there’s a noticeable speed up from things that run a load of short-lived shell scripts. I believe this was also part of the motivation for Ubuntu to use dash as /bin/sh instead of bash. If you require a specific implementation with many non-standard quirks then you can’t move to a different implementation.

                                                                                                                                                                                        And I say all of this as someone who uses bash as their interactive shell.

                                                                                                                                                                                    1. 34

                                                                                                                                                                                      This is just bad functional code to be honest (and all the variations suggested in the article):

                                                                                                                                                                                      const renderNotifications = flow(
                                                                                                                                                                                        map(addReadableDate),
                                                                                                                                                                                        map(addIcon),
                                                                                                                                                                                        map(formatNotification),
                                                                                                                                                                                        map(listify),
                                                                                                                                                                                        wrapWithUl
                                                                                                                                                                                      );
                                                                                                                                                                                      

                                                                                                                                                                                      Here’s what’s bad about it:

                                                                                                                                                                                      1. There’s no clue about what the input to this function is. Point-free style is generally more readable in a language like Haskell, because even when you don’t name the arguments, you can still support the readability with type annotations like renderNotifications :: [Notification] -> SomethingThatIHonestlyCantDiscernFromThisCode. And in a language like Haskell, you can hover your mouse over addIcon in your editor and it will tell you the input and output types (specialized to that call site even when addIcon itself is polymorphic) and if your types are faithful to your domain, that will give you ample clue to understand the code better.
                                                                                                                                                                                      2. Even though it looks like there’s no mutation here, I’d argue that there’s no ease of reasoning gained! We’re passing a list of notifications through 5 stages, and each stage has access to the complete output from the previous stage and it has the power to modify the input to the next stage completely! Isn’t this how we model the semantics of imperative programs to begin with? Each statement in an imperative program is semantically like a function that receives the full state of the program and returns a new state. This means, this code is at least as hard to follow as the equivalent imperative program that mutates the fields of these objects being passed around. But it’s actually much worse, because it goes out of its way to avoid the imperative syntactic sugar and obfuscates it even more.
                                                                                                                                                                                      3. An important goal of functional programming is to build your program up from pieces that require as little context as possible to be meaningful. I’d argue that functions like addIcon are directly in opposition to it. Why don’t you just have an icon here that you pass separately to formatNotification. This way, I can interact with that icon in a REPL or maybe write tests for it without having to create a Notification first. Or maybe the specific icon here depends on the contents of the notification, in that case, why not have a iconForNotification(notification) function with type Notification -> Icon, this way I can pass iconForNotification(notification) separately to formatNotification. That said, in all likelihood, you don’t actually need the entirety of a Notification in order to decide on the Icon, so it’s much better to have a function that accepts that relevant subset of Notification only.
                                                                                                                                                                                      4. This flow thing is also conceptually messy. The easiest way to demonstrate its messiness is to try to assign a type to it. You’ll need very advanced type system features to do that. In Haskell, the equivalent to flow(a,b,c) would be c . b . a, where . has the very simple type (b -> c) -> (a -> b) -> (a -> c). A very good example of things getting conceptually simpler when you break them down.

                                                                                                                                                                                      I think these superficial notions of functional programming are really hurting the adoption of the actual principles that functional programming aims for. It’s not that functional programmers enjoy solving puzzles in the form of writing point-free code or finding ways they can use map, fold or reduce.

                                                                                                                                                                                      The actual tenets are local reasoning and composing your program from small, independently meaningful pieces that can be composed in other ways in order to support new features, isolated tests and the ad-hoc programs you build in a REPL.

                                                                                                                                                                                      1. 9

                                                                                                                                                                                        Very much agree with everything you said here - it’s easy to feel all smug and superior and all your “blub” colleagues are too dumb to understand what you’re doing, but that’s a) not a productive attitude and b) makes you the jackass.

                                                                                                                                                                                        It takes actual insight to distill what makes FP useful in the context of your language/project and make your code fit in in a way that’s a net positive. I’ve very often seen people so enamoured with higher order functions etc that they try to introduce such code into, say, a Python codebase where these things stick out like a sore thumb and are less performant and less debuggable than the “ruthlessly imperative” code that Python tends to encourage.

                                                                                                                                                                                        I also know this because I was the jackass a decade or so ago, trying to force a square peg in a round hole (which was PHP at the time when it not yet had first-class functions). At the time there was no project lead to guide me to the right path. So in a company I’d say it’s important to work with code review and have a senior developer provide feedback on all code, and ensure the code is of a uniform quality.

                                                                                                                                                                                        1. 3

                                                                                                                                                                                          Even though it looks like there’s no mutation here, I’d argue that there’s no ease of reasoning gained! We’re passing a list of notifications through 5 stages, and each stage has access to the complete output from the previous stage and it has the power to modify the input to the next stage completely! Isn’t this how we model the semantics of imperative programs to begin with?

                                                                                                                                                                                          No, it isn’t. These can be written as pure functions with no side effects. The benefit is that you don’t need any more context than the contents of the function to understand what it is doing. Imperative programs have to consider side-effects, and so you require substantially more mental context to understand it.

                                                                                                                                                                                          1. 3

                                                                                                                                                                                            Isn’t this how we model the semantics of imperative programs to begin with?

                                                                                                                                                                                            No, it isn’t.

                                                                                                                                                                                            How do you model the semantics of imperative programs then?

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              An imperative program can be modeled as taking the whole world as input and giving a new version of the whole world as output. Which is not what happens in the functional example.

                                                                                                                                                                                              1. 2

                                                                                                                                                                                                Which is not what happens in the functional example.

                                                                                                                                                                                                I think the point is that it pretty much is what happens in this functional example. Sure, technically we may have referential transparency, but in this example it is no more helpful than the referential transparency you get when you model imperative programs as taking the whole world as input and giving a new version of the whole world as output.

                                                                                                                                                                                                1. 1

                                                                                                                                                                                                  Sure, you do lose the benefits of FP if you assume that the code sample has side-effects and therefore is not doing FP.

                                                                                                                                                                                                  This is a circular argument.

                                                                                                                                                                                                  1. 2

                                                                                                                                                                                                    What? I think it’s clear there aren’t meant to be any side-effects there.

                                                                                                                                                                                                    1. 1

                                                                                                                                                                                                      An imperative program can be modeled as taking the whole world as input and giving a new version of the whole world as output. Which is not what happens in the functional example.

                                                                                                                                                                                                      I think the point is that it pretty much is what happens in this functional example.

                                                                                                                                                                                                      You are saying here that the functional example is mutating shared state and merely disguising this fact with function composition. If that were the case, it would not be functional.

                                                                                                                                                                                                      The entire premise of functional programming is statelessness. The functions are not operating on a shared scope and mutating it in place; they are accepting one type of data, and transforming it into another form as output.

                                                                                                                                                                                                      If the output of one function were routing into multiple other functions, the transformations of the child functions would not impact their siblings. These transformations could take place in parallel without worrying about race conditions.

                                                                                                                                                                                                      Imperative code is stateful; mutations happen within a shared context. Non-deterministic output. Parallelism is dangerous.

                                                                                                                                                                                                      Functional code is stateless. There are no mutations, only copy & transform. Deterministic output. Parallelism is easy and safe.

                                                                                                                                                                                                      1. 3

                                                                                                                                                                                                        You are saying here that the functional example is mutating shared state and merely disguising this fact with function composition.

                                                                                                                                                                                                        No, that’s not what I’m saying at all. No-one else in this thread is saying that either. Here’s the original comment from the beginning of this thread:

                                                                                                                                                                                                        Even though it looks like there’s no mutation here, I’d argue that there’s no ease of reasoning gained! We’re passing a list of notifications through 5 stages, and each stage has access to the complete output from the previous stage and it has the power to modify the input to the next stage completely!

                                                                                                                                                                                                        No-one is claiming that there’s shared state here.

                                                                                                                                                                                                        The point is that we are not gaining the principle benefit of referential transparency: we are not gaining any ease of reasoning.

                                                                                                                                                                                                        1. 1

                                                                                                                                                                                                          I’ve given ample reasons why this is not the case. Pure functions are easier to reason about than imperative functions by definition.

                                                                                                                                                                                          2. 3

                                                                                                                                                                                            There’s no clue about what the input to this function is.

                                                                                                                                                                                            A function that’s named renderNotifications quite clearly takes notifications as input.

                                                                                                                                                                                            Even though it looks like there’s no mutation here, I’d argue that there’s no ease of reasoning gained!

                                                                                                                                                                                            The main advantage of immutability is referential transparency: in its scope, a given symbol always refers to the same value.

                                                                                                                                                                                            Each statement in an imperative program is semantically like a function that receives the full state of the program and returns a new state.

                                                                                                                                                                                            As opposed to a functional program, where a function receives only what you give to it, not the whole world, making it easier to reason about what it can do.

                                                                                                                                                                                            1. 3

                                                                                                                                                                                              A function that’s named renderNotifications quite clearly takes notifications as input.

                                                                                                                                                                                              If that’s a common idiom in the language, maybe. But even then, it’s not obvious that each expression returns modified copies of the input, rather than mutating the input in place. These sorts of ambiguities go away when this kind of thing is written as a sequence of imperative expressions.

                                                                                                                                                                                              This sort of default-immutability, while inarguably useful in many dimensions, makes it very difficult for languages to be competitive in terms of performance.

                                                                                                                                                                                          1. 4

                                                                                                                                                                                            Something is clear to me:

                                                                                                                                                                                            • you have a positive attitude towards functional programming
                                                                                                                                                                                            • your team doesn’t

                                                                                                                                                                                            Both sides rationalize in different directions:

                                                                                                                                                                                            • you think challenging yourself is valuable and the pain it creates is secondary
                                                                                                                                                                                            • your team only sees pain and they look for justifications to hide the fact the decision is already taken

                                                                                                                                                                                            Bottom line: both make a decision based on instinct. The difference is that one enjoys learning while the other don’t, and also cannot tolerate someone who wants to learn.

                                                                                                                                                                                            1. 3

                                                                                                                                                                                              one enjoys learning while the other don’t

                                                                                                                                                                                              As the size of a group of people grows, every property of that group moves toward the mean. Classifications like that one subset of the group enjoys learning and the other doesn’t aren’t IMO productive, because they aren’t metrics that can be influenced by the people writing the code. Engineering can certainly influence hiring, but when you’re writing code, your audience is your colleagues as they are, with all of their strengths and weaknesses and everything else, and not the colleagues that you wish you had.

                                                                                                                                                                                            1. 40

                                                                                                                                                                                              If the people you’re working with tell you there’s “too much magic” there’s too much magic. It’s a subjective position, and the other people on your team are the ones who get to decide. Stop trying to show how clever you are and write code people can read.

                                                                                                                                                                                              1. 22

                                                                                                                                                                                                The opposite also exists. If you are told to write it in a for(…;…;…) loop so that others understand it and you think there are better ways to do it, it’s fine to judge that the team needs some extra maturity.

                                                                                                                                                                                                map, filter and reduce have been existing for a long long time. Using them instead of for loops that construct new arrays is not clever code.

                                                                                                                                                                                                1. 6

                                                                                                                                                                                                  The maturity of the team is not a variable that can be influenced by anyone who is writing code. At least, not in any relevant timeframe. You have to write code to the level of the team you have, not the team you wish you had. Anything else is negligence.

                                                                                                                                                                                                  edit: To a point, of course. And I guess that point is largely a function of language idioms, i.e. the language should be setting the expectations. If some keyword or technique is ubiquitous in the language, then it’s appropriate; if it’s a feature offered by a third-party library then you gotta get consensus.

                                                                                                                                                                                                2. 6

                                                                                                                                                                                                  Stop trying to show how clever you are and write code people can read.

                                                                                                                                                                                                  I think that is a pretty pessimistic interpretation of why people might write code in a particular style. (Though I don’t doubt that’s why some people might do it.) But I think most of the time it’s because they got excited about something and that distinction is important.

                                                                                                                                                                                                  For example, the way you are going to respond to someone who is trying to show off or lord over others is going to be different than someone who is expressing a genuine interest.

                                                                                                                                                                                                  If someone on your team is taking an interest in something new it might be because they are bored. If you treat them like they are being a jerk then you are only going to make them feel worse. Instead, it’s better to redirect that energy. Maybe they need a more challenging project or they should work with an adjacent team.

                                                                                                                                                                                                  That said, someone who is excited about something new might go off the deep end if they are given a lot of freedom and a blank canvas so it’s important to help guide them in constructive and practical directions. Acknowledging interests instead of shunning them can open up space for constructive criticism.

                                                                                                                                                                                                  Overall, it’s important to be kind and try to work together.

                                                                                                                                                                                                  1. 4

                                                                                                                                                                                                    That said, someone who is excited about something new might go off the deep end if they are given a lot of freedom and a blank canvas so it’s important to help guide them in constructive and practical directions.

                                                                                                                                                                                                    Exactly - someone might be so excited about the new possibilities of coding in a certain way they forget about readability in their excitement to golf it down to the most elegant point-free style. But that same programmer, after a few months looking back at their own code might cringe in horror at what they’ve wrought. If even the original author later might not understand what they wrote, it’s a terrible idea to just merge it in even if the other teammembers are total blockheads (which is highly unlikely given the type of work we do), and the tech lead would be in their rights to require modifications to make it more easy to understand.