Threads for barakmich

  1. 15

    How does it handle data structures that are not trees? It big problem with Rust is that it uses linear types as the sole memory management strategy in the language, so anything that requires aliases that have non-lexical lifetimes is impossible to implement. Rust works around this with unsafe blocks that allow you to ignore linearity, but any of these can introduce memory safety bugs. My biggest concern about Rust is that it each one of these unsafe blocks may be locally valid but not safe in the presence of composition because the type system is not expressive enough to capture the invariants that these depend on (the Oakland paper that found 200+ CVEs in Cargo packages looking for three idioms around incorrect use of locally-safe unsafe blocks that I shared here last year adds some weight to that worry). In skimming the article,

    I didn’t see anything that lets me safely implement the kinds of structures that I need in systems programming. For example, a thread object may need to be on a list of threads for a process and on one or more sleep queues. This can’t be implemented with linear types unless you add an indirection layer (and now temporal safety issues still exist, they just result in instance confusion and can still lead to critical CVEs), which adds the kind of overhead that is often unacceptable in systems programming.

    1. 7

      As far as I can tell, Austral is not for you. It is for people who want “simple” Rust, not for people who want more complex Rust.

      I think simplicity is just a deranged goal for a programming language, so I am utterly uninterested in Austral. Austral defines simplicity to be short description length. Why does Austral have if and while statement then? Isn’t goto simpler?

      1. 8

        I think simplicity is just a deranged goal for a programming language

        I don’t know if I agree with this take or not, but it feels so against the current discourse on programming languages that I have no choice but to like it XD

        1. 6

          The word “simple” is being used here with a precise technical meaning, and gotos are not simpler than control structures with single entry and single exit. “Short description length” refers a formal technical description that fully and exhaustively describes the semantics, including in all corner cases.

          1. 4

            A short description is not necessarily simple though. There is no precise meaning to the word “simple.”

          2. 5

            It is for people who want “simple” Rust, not for people who want more complex Rust.

            I don’t want the language to be simpler (in the sense used here). I want using the language to be simpler. Those aren’t the same; Brainfuck and 6502 assembly are good counterexamples.

            One of the main reasons I find Rust hard to use is because it tries to force me into single-ownership patterns. A language like Rust that didn’t have that kind of restriction would be simpler to me, even if its specification were longer.

            1. 1

              Simplicity here has a straightforward definition: it is the amount of information it takes to describe a system.

              That seems like a reasonable thing to optimize for. Imagine if Java tried to follow it…

              edit: However, seems like some of its decisions go against that:

              There are no implicit type conversions anywhere. More generally: there are no implicit function calls. If it’s not in the source code, it’s not happening, and you’re not paying the cost of it.

            2. 6

              Austral has unsafe code, but yuck, using that feature undermines the design goals (safety, ease of reasoning about programs).

              I think the answer to your data structure question may lie in adopting the “data oriented programming” and “entity component system” style of programming that is used for high performance systems programming in video games. I’ve read some articles written by Rust people about exploring this style of programming in Rust.

              The author wants a simple language. Trees and DAGs are simpler than cyclic graphs. One article I read advocating this position called pointers the “gotos” of data structures. But to use this kind of language means abandoning programming idioms we are accustomed to (you said “the kinds of structures I need in systems programming”) and using new idioms that I think we are still in the process of discovering. It’s obvious that Austral is a research project and not a drop-in replacement for the style of systems language you are used to, and I think part of the needed research is developing and perfecting these new idioms.

              a thread object may need to be on a list of threads for a process and on one or more sleep queues

              This structure can be described using a DAG, I think. You don’t need a cyclic graph? You aren’t forced to use strict linear semantics for all resources, there is borrowing, so DAGS can be constructed as long as lifetimes are nested. I think Austral would be forced by its design goals to adopt structured concurrency, where thread lifetimes are strictly nested (unlike in Rust, which doesn’t require this).

              1. 8

                This structure can be described using a DAG, I think

                Correct, but it cannot be expressed with linear pointers because each structure appears on multiple lists.

                You aren’t forced to use strict linear semantics for all resources, there is borrowing, so DAGS can be constructed as long as lifetimes are nested

                That’s the problem. The lifetimes for these structures are not lexical. A thread object remains on a sleep queue for as long as it is sleeping, but that queue may be traversed by another thread (with appropriate synchronisation).

                Some things are unavoidably cyclic. The FreeBSD kernel actually includes a tracing garbage collector because file descriptors passed via UNIX domain sockets can create cycles of the file structures. That problem is unavoidable if you want POSIX semantics (I don’t really, but unfortunately I do want to run a load of code that does).

                1. 3

                  It sounds like you are talking about kernel programming: implementing threads inside a kernel like FreeBSD. I have ideas about that.

                  Austral is very similar to a language I’m developing. In my language, there are pure immutable values (which cannot contain references to mutable objects), and pure functions (which are values), and there are also references to objects containing mutable state, but these references are subject to linear semantics, and are not values. Unlike C (which is used for all modern kernel programming) there is no shared mutable state.

                  The absence of shared mutable state means you must use a very different coding style to write a Unix kernel (in my language). I think it would be a message passing microkernel system. So think about Erlang, or QNX. An unsafe systems language with shared mutable state (like assembly language, C, Zig) must be used to implement the trusted “microkernel” or “language runtime”, central to which is the implementation of threads, and message passing between threads. This microkernel is small, because it is written in an unsafe language, and needs to be carefully audited. The majority of the Unix kernel is written in the safe language. I would want to keep the unsafe microkernel or runtime code separate from the safe code, and maintained by a select group of people, rather than have a coding culture where anybody can drop unsafe code blocks into whatever they are writing. This makes the unsafe code easier to audit and prevents unchecked proliferation. That’s why I talk about using 2 different systems languages, rather than a single language with unsafe blocks.

                  Cyclic structures are implemented using data oriented programming techniques. Instead of a graph of mutable objects allocated on a heap, where the arcs in the graph are machine pointers, you represent a collection of objects as an array or a structure of arrays (SOA), and object references become array indexes or the “handles” of data oriented programming. Data oriented designs are faster, because there are fewer heap allocations and an SOA memory layout can be optimized to provide better cache locality. Some gaming systems languages have language support for SOA and data oriented programming, which makes it easier.

                  1. 4

                    I agree with most of what you say, but to me a systems programming language is one that can be used to implement:

                    • A memory allocator
                    • A scheduler

                    Ideally, it should have very few type-system escape hatcher for this (mostly to some assembly for capturing the register state on context switches) and be able to provide safe idioms for these things.

                    I then see something one step up, where you need fine-grained control over memory lifetimes and communication. This is infrastructure programming, where you might implement device drivers (so you need an atomic model for accessing memory-mapped I/O via typed regions), network protocols (so you need to be able to efficiently parse and generate binary data), but where everything that you do should be expressible within your language’s type system. At this level, you should never need an unsafe escape hatch, you should be able to do absolutely everything in the type system. Linear types at the object level are still not sufficient here because code at this level is very often performance (throughput / latency / jitter) sensitive and so you requiring more complex data structures to express an idea is not adequate.

                    Above this, you have things where programmer experience and efficiency is the primary goal, where you’re happy to put up with some memory (and performance) overhead and some jitter in exchange for reduced cognitive load. This is application programming.

                    For Verona, we are targeting infrastructure programming. The runtime (scheduler, memory allocator) is written in C++, but the only FFI to unsafe code is via a sandboxing model, which prevents the foreign code from being able to corrupt the safe-language data structures. We build a concurrency model into the language, because you cannot implement a concurrency model in a safe language (by definition, it requires shared mutable state). Objects are all allocated in a region. Regions have a sentinel object that dominates all live objects within that region. Pointers to the sentinel from outside are linear, so we can safely move a region between threads by assigning that pointer (or move-capturing it in a lambda) but within a region we can have arbitrary object graphs. A region can have its choice of memory-management model, for example a traced GC if you want complex cyclical data structures, reference counted if you want DAGs (or simple cyclic data structures, such as doubly linked lists, where you will do explicit cycle braking), linear ownership if you want only trees, or bump allocation with bulk free if everything has roughly the same lifetime. We can also freeze a region, which consumes the linear capability to the sentinel and returns an immutable capability to the same object graph. Immutable objects can be shared between concurrent owners (cowns), our unit of concurrency. Work can be asynchronously scheduled over a set of cowns, allowing things like two-phase commit, which are essential to efficient infrastructure programming but incredibly painful with actor-model programming (our concurrency model is a generalisation of the actor model).

                  2. 1

                    I’m super curious about this too. It’s rough enough in Rust, and maybe there’s a better way forward?

                    if you want POSIX semantics (I don’t really, but unfortunately I do want to run a load of code that does)

                    After trying to actually implement POSIX filesystem sematics once upon a time, I’m stealing this saying for the future. :)

                    1. 4

                      The approach that we’re taking in Verona is to move the linearity up one layer. Every object is in precisely one region (an abstract machine concept, not necessarily a contiguous address range). Every region has a single object that dominates all live (reachable) objects in the region. Pointers from the outside of a region to that sentinel object are linear. This gives you the nice concurrency properties (transferring the linear pointer is an ownership transfer of the whole region) but also lets you have different memory management strategies within a region. A region might use a tracing GC, reference counting, or simple bump allocation and bulk free, depending on the use.

              1. 24

                It feels like Python has been trying to move from 2 to 3 for as long as I’ve been using Python.

                1. 15

                  It moved years and years ago. This news is a distro that had all its scripts in Python 2 finally migrating. They’re quite late to doing it.

                  1. 12

                    A language version $version isn’t dead (or truly in the past) until the code you encounter in the wild, doing useful things, is all newer than $version.

                  2. 11

                    The community and all the big parts of the ecosystem have been “moved” for years. Django, for example, declared Python 2 would not be supported past Django 1.11, which was initially released in April 2017. That release got bugfix support until the upstream Python 2 EOL, and then that was that.

                    So what you’re seeing now is not “Python” still “trying to move”. It moved. The author of this post is also pretty clearly in the “what do I do now that I am indisputably being left behind even by the long-support-cycle Linux distros” phase of realizing that maybe things would be better if they had spent even half as much time over the years running the 2to3 tool and cleaning up their code as they’ve spent writing ranty blog posts about Python 3 (though maybe not too much better, since from what I recall of earlier posts their flagship internal application had no test suite, and thus no way to be confident that any port – to Python 3, to another language, to anything – had succeeded).

                    1. 2

                      Eh, cut the guy some slack. If he’s working at a university (as it appears) then academics around him are notorious for (a) not writing tests and (b) not writing things that are easy to maintain. If it works today, gets the grade or degree, and never works again, then “who cares?”, right?

                      (I am definitely not advocating that point of view, but it’s common)

                      1. 2

                        From what I understand, it’s not “academic” (i.e., research) code, it’s some kind of web application that his team maintains for use by the academics.

                        Edit: yeah, this is a description of it. Though I think I recall reading that there’s also some kind of email application they maintain, too.

                      2. 2

                        The author of this post is also pretty clearly in the “what do I do now that I am indisputably being left behind even by the long-support-cycle Linux distros” phase of realizing that maybe things would be better if they had spent even half as much time over the years running the 2to3 tool and cleaning up their code as they’ve spent writing ranty blog posts about Python 3

                        I guess this was a rhetorical device, but I see no indication that the author regrets time spent critiquing Python 2 and not porting to Python 3.

                        1. 2

                          Partly so, yes. Though his writing, over the years – and I do mean that – in which he’s been doing this has evolved in interesting ways, with such evolutions tending to correspond to major milestones at which maintaining working Python 2 code became significantly more difficult (distros switching to Python 3 as the default Python, upstream EOL of Python 2, now distros starting to not even have an optional Python 2 interpreter).

                          It really does feel, to me, like it has gone from being dismissive and rant-y to increasingly despairing as the ecosystem moved on.

                          1. 2

                            I said the author is “pretty clearly” in the “what do I do now” phase – they’re running out of ways to get a working Python 2 interpreter.

                            1. 1

                              Fair enough; obviously regret and despair are different things, and saying the author “pretty clearly” regrets not moving to Python 3 is quite different from “it feels to me” like he is increasingly despairing.

                              1. 1

                                I got my replies crossed up, and replied to you here.

                        2. 4

                          It has for me, and I’ve been using Python for around 12 years now. I’m half expecting a “Python 4” to be announced once the world has finally ditched Python 2, just so we can do the decades-long dance all over again.

                          1. 6

                            In all seriousness, when he was BDFL, Guido said they’d never do another language release comparable to the 2-3 transition again, and while he’s not BDFL anyore, I think it’s safe to assume that opinion is widely shared.

                          2. 3

                            Is it really “Python” who has been trying? I think its programs and applications. And most of them did not really try. Chrome’s build system was Python 2 only until recently. In my experience people regularly underestimate the time required to deprecate and finally remove something. My rule of thumb is: Keep deprecated for multiple years (5-15), then when you think it is time to remove it, wait another few years (e.g., 3).

                            1. 1

                              You’re not alone. I completely ignored Python2 years ago when starting to learn Python(3).

                            1. 21

                              Oh is it time to hype dsls again? That makes sense as we’re starting to all get a little embarrassed about the levels of hype for functional programming.

                              I guess next we’ll be hyping up memory safe object oriented programming.

                              1. 16

                                I’m just sitting here with my Java books waiting for the pendulum to swing back…

                                1. 9

                                  I’m going to go long on eiffel books.

                                  1. 6

                                    I think a language heavily inspired by Eiffel, while fixing all of its (many, many) dumb mistakes, could go really far.

                                    1. 2

                                      I’ve just started learning Eiffel and like what ive seen so far, just curious what do you consider its mistakes?

                                      1. 8
                                        1. CAT-calling
                                        2. Bertrand Meyer’s absolute refusal to use any standard terminology for anything in Eiffel. He calls nulls “voids”, lambdas “agents”, modules “clusters”, etc.
                                        3. Also his refusal to adopt any PL innovations past 1995, like all the contortions you have to do to get “void safety” (null safety) instead of just adding some dang sum types.
                                      2. 1

                                        Racket!

                                  2. 14

                                    I, personally, very much doubt full on OOP will ever come back in the same way it did in the 90s and early 2000s. FP is overhyped by some, but “newer” languages I’ve seen incorporate ideas from FP and explicitly exclude core ideas of OOP (Go, Zig, Rust, etc.).

                                    1. 5

                                      I mean, all of those languages have a way to do dynamic dispatch (interfaces in Go, trait objects in Rust, vtables in Zig as of 0.10).

                                      1. 13

                                        And? They also all support first-class functions from FP but nobody calls them FP languages. Inheritance is the biggest thing missing, and for good reason.

                                        1. 12

                                          This, basically. Single dynamic dispatch is one of the few things from Java-style OO worth keeping. Looking at other classic-OO concepts: inheritance is better off missing most of the time (some will disagree), classes as encapsulation are worse than structs and modules, methods don’t need to be attached to classes or defined all in one batch, everything is not an object inheriting from a root object… did I miss anything?

                                          Subtyping separate from inheritance is a useful concept, but from what I’ve seen the world seldom breaks down into such neat categories to make subtyping simple enough to use – unsigned integers are the easiest example. Plus, as far as I can tell it makes most current type system math explode. So, needs more theoretical work before it wiggles back into the mainstream.

                                          1. 8

                                            I’ve been thinking a lot about when inheritance is actually a good idea, and I think it comes down to two conditions:

                                            1. The codebase will instantiate both Parent and Child objects
                                            2. Anything that accepts a Parent will have indistinguishable behavior when passed a Child object (LSP).

                                            IE a good use of Inheritance is to subclass EventReader with ProfiledEventReader.

                                            1. 10

                                              Take a cookie from a jar for using both LSP and LSP in a single discussion!

                                              1. 4

                                                Inheritance can be very useful when it’s decoupled from method dispatch.

                                                Emacs mode definitions are a great example. Nary a class nor a method in sight, but the fact that markdown-mode inherits from text-mode etc is fantastically useful!

                                                On the other hand, I think it’s fair to say that this is so different from OOP’s definition of inheritance that using the same word for it is just asking for confusion. (I disagree but it’s a reasonable argument.)

                                                1. 2

                                                  Inheritance works wonderfully in object systems with multiple dispatch, although I’m not qualified to pinpoint what is it that makes them click together.

                                                  1. 1

                                                    I’ve lately come across a case where inheritance is a Good Idea; if you’re plotting another of your fabulous blog posts on this, I’m happy to chat :)

                                                    1. 1

                                                      My impression is that inheritance is extremely useful for a peculiar kind of composition, namely open recursion. For example, you write some sort of visitor-like pattern in a virtual class, then inherit it, implement the visit method or what have you, and use this to recurse between the abstract behavior of traversing some structure, and your use-case-specific code. Without recursion you have to basically reimplement a vtable by hand and it sucks.

                                                      Well, that’s my only use of inheritance in OCaml. Most of the code is just functions, sum types, records, and modules.

                                                      1. 1

                                                        Forrest for the trees? When you want to create a framework that has default behaviour that can be changed, extended or overridden?

                                                      2. 4
                                                        • obj.method syntax for calling functions — a decent idea worth keeping.
                                                        • bundling behavior, mutable state, and identity into one package — not worth doing unless you are literally Erlang.
                                                        1. 3

                                                          IMO there is a fundamental difference between Erlang OO and Java OO to the point that bringing them up in the same conversation is rarely useful. Erlang actively discourages you from having pellets of mutable state scattered around your program: sure, threads are cheap, but that state clump is still a full-blown thread you need to care for. It needs rules on supervision, it needs an API of some kind to communicate, etc, etc. Erlang is at it’s best when you only use threads when you are at a concurrency boundary, and otherwise treat it as purely functional. Java, in contrast, encourages you to make all sorts of objects with mutable state all over the place in your program. I’d wager that MOST non-trivial methods in Java contain the “new” keyword. This results in a program with “marbled” state, which is difficult to reason about, debug, or apply any kind of static analysis to.

                                                        2. 2

                                                          In all honesty, you sound quite apologetic to what could be arguably considered objectively bad design.

                                                          Attaching methods to types essentially boils down to scattering data (state) all over the code and writing non pure functions. Why honestly cannot understand how anyone would think this is a good idea. Other than being influenced by trends or cults or group thinking.

                                                          Almost the same could be said about inheritance. Why would fiting a data model in a unique universal tree be a good idea? Supposedly to implicitly import functionality from parent classes without repeating yourself. Quite a silly way to save a line of code. Specially considering the languages that do it are rather verbose.

                                                          1. 5

                                                            Why honestly cannot understand how anyone would think this is a good idea. Other than being influenced by trends or cults or group thinking.

                                                            Here’s a pro tip that has served me well over many years. Whenever I see millions of otherwise reasonable people doing a thing that is obviously a terribly stupid idea, it is always a lack of understanding on my part about what’s going on. Either I am blind to all of the pros of what they are doing and only see the cons, or what they’re doing is bad at one level but good at a different level in a way that outbalances it, or they are operating under constraints that I don’t see or pretend can be ignored, or something else along those lines.

                                                            Billions of lines of successful shipped software have been written in object-oriented languages. Literally trillions of dollars of economic value have been generated by this software. Millions of software developers have spent decades of their careers doing this. The though that they are all under some sort of collective masochistic delusion simply does pass Hanlon’s Razor.

                                                            1. 1

                                                              To be honest, the more I study OOP (or rather, the hodgepodge of features and mechanisms that are claimed by various groups to be OOP), the less room I see for a genuine advantage.

                                                              Except one: instantiation.

                                                              Say you have a piece of state, composed of a number of things (say a couple integers, a boolean and a string), that represent some coherent whole (say the state of a lexer). The one weird trick is that instead of letting those be global variables, you put them in a struct. And now you can have several lexers running at the same time, isn’t that amazing?

                                                              Don’t laugh, before OOP was popular very prominent people thought it was a good idea to have global state in Lex, Yacc, or error handling (errno). So here’s my current guess: the success we attribute to OOP doesn’t really come from any of its overly hyped features. It comes from a couple very mundane, yet very good programming practices it adopted along the way. People attributed to the hyped stuff (such as inheritance) a success they have earned mostly by avoiding global variables.

                                                              Abstract data types are amazing, and used everywhere for decades, including good old C. The rest of OOP though? Contextual at best.

                                                              1. 1

                                                                It has been the opposite for me.

                                                                • typecast everything to and from object in early versions of java
                                                                • EJBs 2
                                                                • Bower package manager. Its creator wrote on stack overflow that he was confused when he created the project and that it was essentially useless.
                                                                • Ruby gems security incident
                                                                • Left pad fiasco
                                                                • Apache web server htaccess configs

                                                                I could go on with more esoteric examples to an ever growing list.

                                                                All these had criticism screaming long before they happened: why?

                                                              2. 3

                                                                Many decisions are only clearly good or bad in retrospect.

                                                            2. 6

                                                              Inheritance is the biggest thing missing, and for good reason.

                                                              That reason being “inheritance was the very first mechanism for subtyping, ADTs, and code-reuse, and people using it got ideas for better mechanisms from it.” ;)

                                                              1. 1

                                                                Exactly!

                                                              2. 3

                                                                The first versions of Simula and Smalltalk didn’t have inheritance either. Self and other prototypal object-oriented languages don’t use traditional inheritance either. We still call all of them object-oriented.

                                                                Honestly, it’s well beyond time that we retire all programming language paradigm terms. Modern languages simply aren’t organized into paradigms they way older simpler languages were.

                                                                It’s like we’re looking at a Honda Accord and arguing over whether it’s a penny farthing or a carriage. The taxonomy no longer makes sense.

                                                            3. 1

                                                              Ah yes and that’s why it’s ripe to have a come back. :)

                                                              Seriously though I expect that the next incarnation will be “oop without inheritance” or something. Probably combined with some large corporation “inventing” gc-less memory management.

                                                              1. 2

                                                                The good parts of OOP never really left. We already have that exact language: Rust. It has formal interfaces (Traits), encapsulation, polymorphism, and gc-less memory management.

                                                                1. 10

                                                                  The main thing about OOP that needs to die is the idea that OOP is a coherent concept worth discussing on its own. Talk about the individual concepts as independent things! It’s much more productive.

                                                                  1. 1

                                                                    Talk about the individual concepts as independent things!

                                                                    IMO OOP these days really means inheritance and an object lifecycle. All the other concepts aren’t really unique to OOP.

                                                                    1. 3

                                                                      I think “OOP” generally means “features of object-oriented languages that I don’t like” to a lot of people. The people using those languages don’t generally get into paradigm arguments.

                                                                      (Personally, I consider inheritance to be common in OOP languages but not a particularly interesting or salient part of them. Many early OOP languages didn’t have inheritance and prototypal ones have an entirely different code reuse model.)

                                                                      1. 1

                                                                        For some people “OOP” means “features of languages I do like”. For instance I’ve seen people include templates/generics/parametric polymorphism and unnamed functions as core parts of OOP… having learned CamlLight (OCaml without the “O”) in college, I confessed I was quite astonished.

                                                                      2. 2

                                                                        You say that but it means different things to different people. I don’t disagree that your definition would be a good one if you could get people to agree on it, but I can’t assume that when other people say “OOP” that’s what they’re talking about.

                                                                2. 1

                                                                  I think it will come back, rediscovered as something new by a new generation disillusioned with whatever has been the cool solves-everything paradigm of the previous half decade. Perhaps this time as originally envisaged with a “Scandinavian school” modeling approach.

                                                                  Of course it never left as the first choice for one genre of software… the creation of frameworks featuring default behavior that can be overridden, extended or changed.

                                                                  Those languages you mention (Go, Zig, Rust) are primarily languages solving problems in the computer and data sciences, computing infrastructure and technical capability spaces. Something is going to be needed to replace or update all those complex aging ignored line-of-business systems.

                                                                3. 11

                                                                  There isn’t really any need to “hype” DSLs because they’re already widely used in all domains of programming:

                                                                  • front end: HTML / CSS / JavaScript, and most JS web frameworks introduce a new DSL (multiple JSX-like languages, Svelte, etc.)
                                                                  • back end: a bajillion SQL variants, a bazillion query languages like Redis
                                                                  • builds: generating Ninja, generating Make (CMake, Meson, etc.)
                                                                    • there at least 10 CI platforms with their own YAML DSLs, with vars, interpolation, control flow, etc.
                                                                  • In games: little scripting languages for every popular game
                                                                  • Graphics: scene description languages, shader languages
                                                                  • Compilers: LLVM has its own TableGen language, languages for describing compiler optimizations and architecture (in the implementation of Go, a famously “not DSL” language), languages for describing VMs (Ruby)
                                                                  • Machine Learning: PyTorch, TensorFlow, etc. (these are their own languages, on top of Python)
                                                                  • Distributed computing: at least 10 MapReduce-derived frameworks/languages; there are internal DSLs in Scala for example, as well as external ones
                                                                  • Mathematics and CS: Coq, Lem, etc.

                                                                  All of these categories can be fractally expanded, e.g. I didn’t mention the dozens of languages here: https://github.com/oilshell/oil/wiki/Survey-of-Config-Languages – many of which are commonly used and featured on this site

                                                                  If you think you don’t use DSLs, then you’re probably just working on a small part of a system, and ignoring the parts you’re not working on.

                                                                  ALL real systems use tons of DSLs. I think the real issue is to mitigate the downsides

                                                                  1. 1

                                                                    Oh yes but at the same time if you haven’t seen the hype for DSLs then you haven’t spent long enough in the industry to go through that part of the hype cycle. DSLs are what they are and it looks like we might be entering a hype cycle where people want to make them out to be much more.

                                                                    1. 3

                                                                      I don’t agree, I’ve been in the industry for 20+ years, there are plenty of things more hyped than DSLs (cloud, machine learning, etc.)

                                                                      DSLs are accepted standard practice, and widely used, but often poorly understood

                                                                      I’m not getting much light from your comments on the subject – you’ve made 2 claims of hype with no examples

                                                                      1. 2

                                                                        Here’s an example of recent hype https://www.codemag.com/Article/0607051/Introducing-Domain-Specific-Languages

                                                                        Here’s some hype from the year 2000 https://www.researchgate.net/publication/276951339_Domain-Specific_Languages

                                                                        Arguably the hype for 4GLs was the prior iteration of that specific hype.

                                                                        I’m not arguing that DSLs are bad - I’m saying that they’re one of the things on the roster of perfectly good things that periodically get trumpeted as the next big thing that will revolutionize computing. These hype cycles are characterized by attempts to make lots of DSLs when there isn’t a strong need for it or any real payoff to making a language rather than a library.

                                                                  2. 4

                                                                    I know it might sound a bit controversial, but the way I see it we need to reach a new level of abstraction in order for large-scale software development to be sustainable. Some people might say AI is the way forward, or some other new programming technique. Either way I don’t think we’ll get there by incrementally improving on the paradigms we have—in order to reach the next level we’ll have to drop some baggage on the way up.

                                                                    1. 4

                                                                      I mean, humans aren’t getting better at groking abstraction, so I don’t know that “new levels of abstraction” are the way forward. Personally, I suspect it means more rigor about the software development process–if you’re building a tall tower, maybe the base shouldn’t be built with a “move fast and break things” mentality.

                                                                      1. 3

                                                                        Groking abstractions isn’t the problem, at the end of the day abstractions are just making decisions for the users of an abstraction. Over-abstraction is the root of many maintainability woes IMO, the more a programmer knows what’s actually going on underneath the better, but only to the degree that it’s relevant.

                                                                      2. 3

                                                                        I’ve heard it before. DSLs have their place, and some people love them while others hate them. This is one of a rotating cast of concepts that you’ll eventually see rehyped in 10 years.

                                                                    1. 3
                                                                      1. Warning: arrow-key selection will result in wrong guesses

                                                                      2. Out of paranoia, just in case GitHub does some kind of rate limiting by checking the referrer, I set referrerPolicy: ‘no-referrer’ on my requests.

                                                                        Yes, @healeycodes, you are the baddies.

                                                                      1. 2

                                                                        On (2)… wow, no kidding. How hard is it to even just literally check out the top 100 repos and index them on your own service, thus eliminating any API calls at all?

                                                                      1. 1

                                                                        This is dang cool, but IMO the untapped win here is (a) a stacked-commit SCM (which this is, yay!) with a concurrently developed (b) self-hostable, repo-host-agnostic code review tool built for stacked commits. So open source ReviewStack and you’ve got a stew going!

                                                                        1. 15

                                                                          Yeah… that title is misleading.

                                                                          Sure, containers images are tarballs (as they should be) and chroot is part of the process. But there’s also a lot more going on, which you allude to but kind of dismiss.

                                                                          If you want a deeper version of this blog concept written a long time ago by an expert, https://ericchiang.github.io/post/containers-from-scratch/

                                                                          1. 6

                                                                            This is, incidentally, a thing I dislike a lot about Rust stylistically. A lot of Rust is written chaining a bunch of map()s and or_else()s and such.

                                                                            It’s passable if, even as in this article, it’s a straight transform. But it rarely is. There’s often side effects (either unintentional bad ones or good ones that’d make life simpler (eg, a max/min, without using tuples on tuples))… or implicit awaiting in async version of this chaining (how many things are happening in parallel? Hope you did the right buffering call!) and… it’s just a beast to debug. A for loop would be way more obvious in a lot of cases (if for no other reason than to crack open gdb)

                                                                            1. 6

                                                                              In my experience there’s a lot of cases when writing Rust code where you have a simple transform and do want to write it using chained maps and or_else (or even more concisely with the ? operator). When what you’re doing isn’t actually a simple transform, it’s definitely useful to resort to a non-functional construct like a for-loop, but that’s ideally the heavier and less common thing to do.

                                                                              1. 2

                                                                                Why would a for loop be “heavier” than a chain of transforms?

                                                                                If anything, the for loop is easier to optimize by the compiler, and easier to understand by the layperson, right?

                                                                                1. 1

                                                                                  I have no idea whether iterator chains or a for-loop is easier to optimize by the compiler - I’ve seen deep-dives into rustc compiler internals that have argued that iterator chains are actually faster at least sometimes, and I think the broader problem is that it’s difficult for a programmer to actually know which of several semantically-equivalent ways of writing a program will actually result in the most performant code.

                                                                                2. 1

                                                                                  This lines up with my Java intuition as well, although there are so many different Java styles that I don’t claim it’s a universal thing other Java programmers would agree with. If something is doing explicit for loops to transform a collection into another collection, my assumption is either: 1) it’s legacy pre-Java-8 code, written before java.util.stream existed, or 2) it’s doing complex or wonky enough logic that it doesn’t map nicely onto the standard operations, so really does need to drop down to a custom loop.

                                                                                3. 4

                                                                                  A lot of Rust is written chaining a bunch of map()s and or_else()s and such.

                                                                                  I used to do this a lot. The APIs are there. It’s so tempting when there’s a function like map_or_else that looks like it was designed to solve your exact problem. But I think it’s a trap, and as you start writing it often becomes more complicated than anticipated.

                                                                                  These days, I am more skeptical of the ‘functional’ style in Rust. I rely more on language features like match, ?, traits like From/TryFrom, and libraries like anyhow or serde. I couldn’t tell you why, but this is how I feel after using Rust for a couple years.

                                                                                  1. 3

                                                                                    I agree. Chaining in a single expression is usually less readable than a sequence of independent expressions.

                                                                                    1. 2

                                                                                      Yeah, we write a lot of functional heavily chained code in D, but it’s viable (IMO) because all our value types are immutable. No state, pure functions only. There’s a few keywords that are allowed to do mutation (each, sort), but we know to look out for them.

                                                                                    1. 3

                                                                                      Why not combine it all? Because collisions. Even in Rust, the collision between a tokio::sync::Mutex and a std::sync::Mutex (and some projects, including me, prefer parking_lot::Mutex) and all are subtly different.

                                                                                      So chances are you have a dependency that collides with you. Or with each other. (Though if this means that JS-land will have fewer tiny, crappy dependencies (cough cough leftpad), it may end up being a benefit…)

                                                                                      Now, the argument for full qualification instead of relative paths is a very good one and that is well worth doing.

                                                                                      As per Python, “Namespaces are a honking good idea… use more of them!”

                                                                                      EDIT: full qualification is so good I just refactored my stuff to that, so thank you!

                                                                                      1. 1

                                                                                        Combine everything is a viable approach if the ecosystem chooses it. This is how Ruby works, and it’s fine because people know they need to “own” a module namespace to use it.

                                                                                        1. 1

                                                                                          Addressed your comment RE collisions (updated article), TLDR it is non-issue because all exports are manually mapped, so you can prevent collision by aliasing the exports.

                                                                                        1. 17

                                                                                          then it can help you make your code more concise and readable.

                                                                                          # Reuse result of "func" without splitting the code into multiple lines
                                                                                          result = [y := func(x), y**2, y**3]
                                                                                          

                                                                                          I don’t get it. Is this just to avoid writing one extra line, like this?

                                                                                          y = func(x)
                                                                                          result = [y, y**2, y**3]
                                                                                          
                                                                                          1. 12

                                                                                            The article explicitly addresses this objection:

                                                                                            You might say, “I can just add y = func(x) before the list declaration and I don’t need the walrus!”, you can, but that’s one extra, unnecessary line of code and at first glance - without knowing that func(x) is super slow - it might not be clear why the extra y variable needs to exist.

                                                                                            The walrus version makes it clear the y only belongs to that statement, whereas the “one extra line” version pollutes the local scope, and makes your intention less clear. For short functions this won’t matter much, but the argument is sound.

                                                                                            1. 5

                                                                                              with y = func(x):

                                                                                              1. 5
                                                                                                for i in range(3):
                                                                                                    print(i)
                                                                                                print(i)  # prints 2
                                                                                                

                                                                                                Local scope gets polluted all the damn time in Python. I’m not saying that’s desirable, but it is part of the language, and that’s a worse argument then most.

                                                                                                y with the walrus, as written in the article, pollutes local scope anyway, by the by, as the result of func(x). No intentionality is lost and you’ve done literally the same damn thing.

                                                                                                Do what you can’t do and you’ve got my attention. (For my earlier post, using it within an any, at least, takes advantage of short-circuiting (though implicit!))

                                                                                                1. 1

                                                                                                  y with the walrus, as written in the article, pollutes local scope anyway, by the by

                                                                                                  You are correct, I should have tested myself before saying that. So yeah, the benefits look much weaker than I had supposed.

                                                                                              2. 10

                                                                                                Yeah, I find the walrus operator redundant in almost every case I’ve seen it used. If I’m feeling generous, I’ll give partial credit for the loop-and-a-half and the short-circuiting any behavior— but it was wrong to add to the language and I’ve banned it from any Python I’m in charge of.

                                                                                                Edit: Also, the accumulator pattern as written is horrendous. Use fold (or, yes, itertools, fine) as McCarthy intended

                                                                                                1. 9

                                                                                                  You can make the same argument with only minor modification of the examples against many things that now are considered not just acceptable but idiomatic Python.

                                                                                                  For example, you always could just add the extra line to manually close a file handle instead of doing with open(some_file) as file_handle – so should we ban the with statement?

                                                                                                  You could always implement coroutines with a bit more manual work via generators and the yield statement – should we ban async and await”?

                                                                                                  You could always add the extra lines to construct an exception object with more context – should we ban raise from?

                                                                                                  Hopefully you see the pattern here. Why is this bit of syntax suddenly such a clear hard line for some people when all the previous bits of syntax weren’t?

                                                                                                  1. 12

                                                                                                    I once had a coworker that felt that functions were an unnecessary abstraction created by architecture astronauts. Everything people did with functions could be accomplished with GOTO. In languages without a GOTO statement, a “professional” programmer would write a giant while loop containing a series of if statements. Those if statements then compared with a line_number variable to see if the current line of code should be executed. The line_number was incremented at the end of the loop. You could then implement GOTO simply by assigning a new value to line_number. He argued that the resulting code was much more readable than having everything broken up into functions, since you could always see what the code was doing.

                                                                                                    1. 3

                                                                                                      “If the language won’t let us set the CPU’s IP register, we’ll just make our own in memory!”

                                                                                                      Yikes.

                                                                                                    2. 1

                                                                                                      with is a dedicated statement. It does 2 things ok, but only them, no more. You can’t add complexity by writing it in another complex structure like a list-comprehension. The walrus can be put everywhere, it’s vastly different IMHO.

                                                                                                  2. 4

                                                                                                    I also find that in result = [y, y**2, y**3] it’s much clearer to parse what’s going on at a quick glance, while you need to think more when coming up on the walrus.

                                                                                                    Even clearer might be result = [y**1, y**2, y**3].

                                                                                                  1. 20

                                                                                                    TLS probably has a lot of impact on this. First, TLS handshake requires at least two exchanges that need to be ACKed before sending the actual HTTP response. In that time TCP might figure out the bandwidth. Or if not, spent most of those initial 10 packets leaving almost nothing for HTTP.

                                                                                                    1. 7

                                                                                                      It does seem really weird that this is only barely mentioned in the article when it’s overwhelmingly likely to dominate any considerations having to do with slow-start.

                                                                                                      1. 6

                                                                                                        Exactly this. Terminating TLS has a real cost and eats up the budget that you’re talking about. I can’t stand this sort of posturing bull that ignores reality. The original author should go funroll some loops.

                                                                                                      1. 5

                                                                                                        Hiya! Your friendly neighborhood graph database author here, here to fill in a couple comments on what I’ve experienced in the semantic web world over the years.

                                                                                                        First off, really well done article. “This is why OWL is such a bear” made me laugh a hearty D&D laugh. To some points though:

                                                                                                        • I agree that “not having a Unique name assumption is a mistake” – very strongly. It’s why I generally recommend that everything in the world starts with a random UUID (forcing a unique name) and we only then start to explicitly say what, in the devil, this entity is; we can add URIs, other synonymous UUIDs, etc, but we do so explicitly. This has a large operational cost, however. If I have two datasets UUID-A is-a person; UUID-A born-on 1961-08-04 and UUID-B is-a POTUS; UUID-B name "Barack Obama" and then I add the link UUID-A sameas UUID-B – querying between equivalent IDs to find, say, birthdays of presidents, requires either a complete inference step on every traversal (to find sets of equivalent IDs) or a potentially unbounded reindexing operation on the addition of sameas links. You can see why this gets hairy on the DB side.
                                                                                                        • I disagree that the Open World Assumption is ‘serious mistake’ – more neutral. (I agree that having no uniqueness and an open world assumption both together is bad). I see it as kind of a “what sort of query do you want?” question – it’s the two sides of the incompleteness theorem and in some cases you’d want one or the other for your reasoning. Either my reply is consistent, along with inferences, but it may be incomplete– or my response is complete (total over the data I have), but existing data takes precedence, and so I may be slightly inconsistent. I’ve always imagined that, with a really good inference engine, I could flip this assumption at will at query time (and it would infer things somewhat differently as a result, particularly when it comes to universal/existential quantifiers) – but in general, I still think the default state of that switch should probably toward the OWA.
                                                                                                        • OWL is a mess. It’s well-intentioned – you want to be able to write inference rules in the schema data itself, I think we’d all agree – but the semantics of OWL (so, meta-semantics) are not at all friendly or intuitive. A stripped down OWL like you’d suggest (your subclass example, simply complaining, is a good one) would probably be an open opportunity. :)

                                                                                                        But OWL being a mess is farther down the iceburg. RDF itself is a mess. You started talking about it from a format perspective, but there’s a big content gap too between RDF and JSON. If I could write an RDF 2.0 spec tomorrow, I’d take the 1.1 spec, evaluate if there’s anything I’d particularly like to change or rip out (probably a few around object value typing and defining ‘subgraphs’ – more on that in a minute) and, in the name of sheer pragmatism, add only the following things:

                                                                                                        1. An index field. I think JSON got right the minimal set of types you need to get by – numbers (hopefully real ints and floats as opposed to JS “numbers” though), bools, strings, maps (objects), slices (arrays) and… yeah, that’s about it. Of all these, the most painful in RDF is the array. To have an ordered set of objects, ie, a list, in RDF, they act as a linked list, with next and prev links. This is hella inefficient and just weird for most use cases, and simply having an optional index field to set an ordering for a given predicate would be amazing. If it were:
                                                                                                        my-grocery-list has-item [1] milk
                                                                                                        my-grocery-list has-item [0] apples
                                                                                                        my-grocery-list has-item [2] bread
                                                                                                        

                                                                                                        instead of

                                                                                                        my-grocery-list has-item milk
                                                                                                        my-grocery-list has-item apples
                                                                                                        my-grocery-list has-item bread
                                                                                                        apples next-item milk
                                                                                                        milk next-item bread
                                                                                                        

                                                                                                        I’d be a happy camper

                                                                                                        1. Arbitrary KV pairs on links. Neo4j folks love to talk about being a “property graph” and they have half of a point. You probably don’t want arbitrary KV pairs on your nodes – this provides two ways of doing the same thing and one is less powerful. Sure, I could have a node /en/barack_obama{name: "Barack Obama"} but why not just make that a normal, everyday name edge pointing to a literal string, ie /en/barack_obama name "Barack Obama"? It’s kinda what we’re already building for.

                                                                                                        KVs on edges however, I can get behind. Fun fact, it’s harder than you’d think to represent an arbitrary filesystem in RDF, if you take into account hard links (without adding loads of indirection). Try it:

                                                                                                        mkdir foo
                                                                                                        cd foo
                                                                                                        touch bar
                                                                                                        ln bar baz
                                                                                                        

                                                                                                        This graph is, in naive RDF:

                                                                                                        inode-for-foo dirent inode-for-bar
                                                                                                        inode-for-foo dirent inode-for-bar
                                                                                                        

                                                                                                        Twice! Sure, I could use a blank node thing:

                                                                                                        inode-for-foo has-dirent _A
                                                                                                        inode-for-foo has-dirent _B
                                                                                                        _A name "bar"
                                                                                                        _A inode inode-for-bar
                                                                                                        _B name "baz"
                                                                                                        _B inode inode-for-bar
                                                                                                        

                                                                                                        And that would work, but now all my links are two-hop has-dirent->inode instead of the direct contains – anyway, imagine the world where I could annotate those links and get the best of both:

                                                                                                        inode-for-foo dirent {name: "bar"} inode-for-bar
                                                                                                        inode-for-foo dirent {name: "baz"} inode-for-bar
                                                                                                        

                                                                                                        This helps many other scenarios (for what we called “CVTs” back in the day) like marriages and such.

                                                                                                        1. Optionally, canonical IDs for links themselves – This is an alternative to the arbitrary KV pairs on links that can also have interesting properties. Sure, there’s RDFS and RDF* and similar proposed – and actually adopting one of those might do the trick. It’s a discussion to see if it’s better or worse to be able to annotate links with links as opposed to just adding KV fields – I’m kinda six-of-one-half-dozen-of-another about it, but I can see value both ways. The question comes to, are the annotations inherent to the identity of the link (like in the dirent case) or are they something that, in kind of an open-world way, can be added later? (Like, perhaps, I know that this marriage link between two people occurred, but I don’t know when, but if I find out when they got married later, do I rewrite the link entirely or annotate the link?)

                                                                                                        — Those would help a heck of a lot.

                                                                                                        I’ve got more to say on context: ansible-rs makes a very good observation in this comment but I’ll let that simmer a second.

                                                                                                        1. 17

                                                                                                          Between their Github post and this one, I have no respect for the sfconservancy and they all sound like blathering elitists that see everything in black and white with no middle ground or nuance.

                                                                                                          1. 9

                                                                                                            What’s wrong with this post? It correctly points out that selling FOSS through software “stores” is a common way for FOSS projects to make money, and that Microsoft is now removing that source of funding. That’s a real problem for the projects which rely on Microsoft’s store for funding, and it’s not clear why they’re doing it other than to hurt FOSS. It also provides historical context about Microsoft’s bad treatment of FOSS, on their store and otherwise.

                                                                                                            1. 2

                                                                                                              I remember how controversial it was when MuseScore moved to requiring copyright assignment, but it should allow them to dual license and keep it in the Windows store. Expect other projects to at least consider it, and expect a lot of sound and noise around it. Others might just move to Steam, and I wouldn’t be surprised to see some cheeky tweets out of Valve about this.

                                                                                                              1. 1

                                                                                                                Does Steam offer other software than games? Interesting!

                                                                                                                1. 4

                                                                                                                  They do!

                                                                                                                  I use JWildFire to make fractal art. There’s a version on Steam with the runtime bundled: https://store.steampowered.com/app/1445310/JWildfire/

                                                                                                                  Worth the $12, but you can install it free from the developer’s website. Steam has a whole category for software. It’s mostly stuff for developing and modding games, as you would expect, but there’s the occasional oddball with no clear game-related function.

                                                                                                            2. 4

                                                                                                              Hear, hear. They’ve got an axe to grind and they fail to see the other ways in which not having this rule hurts free software as well, as many of these comments point out. Weeding out scams and malicious actors that use open source for a quick buck is probably a net positive.

                                                                                                              1. 1

                                                                                                                Yep. At first their uh, strong tone, felt necessary since their opponents are very strong themselves, but now seeing this I wonder if they’re just…venting? Like it makes them feel really good about themselves because they’re fighting a company… I feel like this particular policy was created by internal employees pushing this to protect users and open source itself. Microsoft is actually losing money on this policy change…

                                                                                                              1. 3

                                                                                                                Shout-out to Bitwarden, which is amazing, supports 2FA, and is self-hostable (but I pay them to support small devs)

                                                                                                                If this change worries you, check it out. Because of Bitwarden, my reaction was “meh, sure, whatever”

                                                                                                                1. 3

                                                                                                                  So I work for a company that generally embraced PACELC early in design. And, while I definitely agree it’s a more nuanced approach to the topic than CAP, both PACELC and CAP suffer from the fact that they’re kind of extrema. Yes, in the limit, you need to choose between these things — and PACELC basically hits the common consensus on CAP that, since you can’t control network partitions in code (someone trips over a network cable!) you’re left choosing between C&A.

                                                                                                                  Once you start really getting into consistency and availability, though, it’s a much more nuanced world. I always refer to https://jepsen.io/consistency where that tree defines many gradients (and trade-offs, and whole ecosystems!) in the Consistency/Availability spectrum.

                                                                                                                  1. 7

                                                                                                                    Snaps are the reason I’ve completely abandoned Ubuntu and never deploy it anywhere anymore. If anything, I’ve gone full Fedora as a result.

                                                                                                                    Canonical does fine work, and I’ve heard all the arguments for ease of maintenance as regards snap-ing things, which I can both appreciate and also point to flaws with (that others have written on better than I can). At the end of the day, it’s another in a litany of Ubuntu-only things (upstart, mir for a minute, that network manager (which wasn’t half bad even)) that seemingly are built in a vacuum.

                                                                                                                    1. 8

                                                                                                                      My guess is that the mypy developers reasonably thought that introducing such a major feature needs to be done gradually. If you see 100 errors when you run mypy the first time, that could be fixable in an afternoon. If you see 10,000, you just give up and stop trying.

                                                                                                                      To get to a good place, start with the default mypy configuration, then work your way up to strict compliance (see mypy --help and apply each of the --strict flags separately), and finally just use mypy --strict, which fixes most of the issues mentioned in this post. Or change to a compiled language.

                                                                                                                      1. 12

                                                                                                                        Yeah, this is the way.

                                                                                                                        I can’t understand why the author (a) wants to use Python, a dynamically typed language (b) wants to use type checking but (c) will give up on type checking that isn’t 100%-guaranteed-enforced

                                                                                                                        • Add mypy to your CI. Congrats, now it’s enforced. There goes whole sections of this post.
                                                                                                                        • Any is an escape hatch because, shocker, Python is dynamically typed (ie, it’s of type PyObject* but shhhh). Its existence doesn’t mean that things, in your own codebase, that are hinted aren’t helpful! Or at least it’s very strange to argue that “anything could be Any, and Any is bad, so let’s not use hints at all, so now everything is Any!”
                                                                                                                        1. 9

                                                                                                                          I have never seen a statically typed language that didn’t have optional dynamic typing. Java has Object, Haskell has Dynamic (and a few other options), C has void*. Any is hardly news to the space

                                                                                                                          1. 4

                                                                                                                            Add mypy to your CI. Congrats, now it’s enforced. There goes whole sections of this post.

                                                                                                                            This is the way.

                                                                                                                            We did this early on and it has done wonders for code quality.

                                                                                                                        1. 31

                                                                                                                          The article doesn’t refute any of the arguments in the blog it’s responding to. Yet it has a point - Gemini seems to have taken off as a subculture.

                                                                                                                          People rally around this protocol as a backlash against the complexity of the web, even though a non-complex website can be made with HTTP and HTML, and it can be read on low-powered computers with oldschool browsers like lynx or slightly more modern links, dillo or netsurf. Gemini is not about any of the technical features (or even the anti-features, if you will) but more about the emotion and ethos surrounding it.

                                                                                                                          1. 11

                                                                                                                            It’s simpler than that: Gemini is about having fun!

                                                                                                                            1. 7

                                                                                                                              Gemini is not about any of the technical features (or even the anti-features, if you will) but more about the emotion and ethos surrounding it.

                                                                                                                              Gemini is an NFT.

                                                                                                                              1. 1

                                                                                                                                I can’t really see the similarity myself.

                                                                                                                                1. 5

                                                                                                                                  It’s a technical solution to a social problem with a committed bunch of supporters who are Extremely Online.

                                                                                                                                  Thankfully there is no money involved however.

                                                                                                                              2. 7

                                                                                                                                It does address the charge of exclusionism. I would even respond to that charge in stronger terms: if you require every new project to be 100% inclusive from day 1, you’re a useful idiot for rich monopolists who have large departments devoted to marketing how inclusive and accessible they are[1].

                                                                                                                                Reading pages on Gemini requires installing a program. We all used to do this, back in the day! It blows my mind that this is considered exclusionist.

                                                                                                                                [1] Except they’re not really if you actually focus on the details. Why the fuck is the author of that other blog post not complaining about how exclusionist Twitter is? They can’t render 280 fucking characters without Javascript.

                                                                                                                                1. 18

                                                                                                                                  Reading pages on Gemini requires installing a program. We all used to do this, back in the day! It blows my mind that this is considered exclusionist.

                                                                                                                                  People install apps on their phones and laptops all the time (lol you should see my “Messenger” app group.) Gemini isn’t exclusionist for asking for an app install, it’s exclusionary for being mostly text based, for emphasizing keyboard navigation, and for eschewing accessibility for minimalism. If I were ever interested in sharing math on Gemini it would be pretty much impossible; there’s no way to represent the markup. In theory you can share other formats, like HTML or images, but in practice the community strongly wants to stick to text/gemini.

                                                                                                                                  There’s also a decent amount of purity politics. See https://github.com/makeworld-the-better-one/amfora/issues/199 for example. It’s a set of cultural values that wants to exclude and circumscribe by default. There’s nothing wrong with this, but it makes the community by definition exclusionary.

                                                                                                                                  1. 10

                                                                                                                                    There’s also a decent amount of purity politics. See https://github.com/makeworld-the-better-one/amfora/issues/199 for example.

                                                                                                                                    I’ve been intrigued by the idea behind Gemini for awhile now (and Gopher before that) but reading through that conversation just made me absolutely certain that I never want anything to do with Gemini. To be fair, I now also doubt that they would want me involved in their community either :-)

                                                                                                                                    1. 14

                                                                                                                                      I mean, it’s mainly just Drew Devault who’s the issue in that exchange. If it makes you feel any better, he’s already been banned from Lobsters :)

                                                                                                                                      1. 1

                                                                                                                                        I haven’t been here for a few months so this is news to me. Whoah.

                                                                                                                                      2. 9

                                                                                                                                        That github issue made me so angry that I added a gemini title grabber to my IRC bot that would also fetch the favicon.txt file, just to spite that asshole.

                                                                                                                                      3. 4

                                                                                                                                        Yeah, that’s totally valid. “Accessible” is a more precise keyword here than “inclusive” that isn’t talked about in either OP or the post it’s responding to. It’s true that plain text isn’t accessible.

                                                                                                                                        I’ve been agonizing about this because my project is terminal-based. I’d characterize my current position as very/genuinely reluctantly excluding some people. I’d love to brainstorm ways to get around it with an accessibility expert.

                                                                                                                                        1. 8

                                                                                                                                          I’d love to brainstorm ways to get around it with an accessibility expert.

                                                                                                                                          Yeah it’s something I’m a bit sensitive to because I have some really bad RSI issues. Personally, I’ve always learned much better with text than drawings (since I was a child), and when I found text interfaces on computers, I found them much easier to navigate around than graphical interfaces. Unfortunately I had a sports injury when I was young in my wrist, and years of coding have now made my RSI pretty bad. There are days when I get by with using only my left hand on an ambidextrous trackball. Those days using the terminal is a gigantic chore and I feel super bummed when I read the fashionable online TUI maximalism in tech spaces. And I’m relatively lucky and privileged, I wasn’t even born with an actual disability. I can only imagine what it’s like for folks with other accessibility issues.

                                                                                                                                          I recall in the ’90s (though I may be conflating trends, so this might be more of a haphazard connection than a true connection) a desire to have the Web contain text and rich media to accommodate the information acquisition style most beneficial to the reader/viewer. By ideologically sticking to text the way Gemini does, I see Gemini making a strong statement that Geminauts should learn a certain way and that other types of learners (say graphical or aural learners) are not really considered. The Web as an open, welcoming technology then feels very different than the closed, specific community of Gemini.

                                                                                                                                          That said, as a personal project, you can’t “fix the world”. Focusing on a niche is fine IMO. We all have finite time in our lives and we do what we can with our time.

                                                                                                                                          1. 8

                                                                                                                                            other types of learners (say graphical or aural learners

                                                                                                                                            Just as a side note here: the idea that people have a “learning style” and one person will learn best with audio vs another best with visuals, has been widely refuted.

                                                                                                                                            1. 3

                                                                                                                                              Want to +1 your comment. I’m aware but I didn’t add that into my post and I don’t want folks to think that my statement is a statement on the pedagogy at large on education.

                                                                                                                                            2. 2

                                                                                                                                              Thanks! If you ever get around to trying out my thing, I’d love to hear any advice you have, whether on accessibility or anything else.

                                                                                                                                            3. 1

                                                                                                                                              There’s a big difference between a terminal-based application for people to explore and ‘the web is for normies and if you don’t join our clique “you’re a useful idiot for rich monopolists”’ – which is a rather exclusionary thing to say.

                                                                                                                                              1. 1

                                                                                                                                                I don’t actually use Gemini much! I don’t hang out on the mailing list, I don’t know anybody there. If I’m in a clique, it’s a clique of 1.

                                                                                                                                                If you require every new project to be 100% inclusive from day 1, you’re a useful idiot for rich monopolists who have large departments devoted to marketing how inclusive and accessible they are

                                                                                                                                                I stand by this statement in all particulars. But I’m not sure who I’m excluding from what by saying it.

                                                                                                                                                1. 5

                                                                                                                                                  If you require every new project to be 100% inclusive from day 1, you’re a useful idiot for rich monopolists who have large departments devoted to marketing how inclusive and accessible they are

                                                                                                                                                  That statement implicitly assumes that projects will later be extended to be more inclusive.

                                                                                                                                                  The problem is, Gemini seems pretty hostile to any extensions made after day 1, and this seems to be a specific goal of the project. This means if they ever want to include accessibility, it has to be planned in from day 1.

                                                                                                                                                  1. 2

                                                                                                                                                    I’m not sure what accessibility it needs to include? It’s trivial to create an audio-only, reading only, braille-only, large print, high contrast, translated to any language, version of any gemini capsule. Lacking support for mathematical notation or music isn’t about accessibility, it’s about content.

                                                                                                                                                    1. 5

                                                                                                                                                      Questions I’d have:

                                                                                                                                                      • How would a screen reader know what language a capsule is? What if you use multiple languages in your capsule?
                                                                                                                                                      • How does a screen reader know what to parse and what not to (e.g. images)?
                                                                                                                                                      • Those Figlet ASCII art images are absolute garbage for a screen reader

                                                                                                                                                      It’s trivial to create an audio-only, reading only, braille-only, large print, high contrast, translated to any language, version of any gemini capsule

                                                                                                                                                      So it’s been said since the beginning of the protocol but all I’ve seen is TUI clients, an Emacs client, and a handful of GUI clients which are still not thinking about accessibility at all.

                                                                                                                                                      1. 5
                                                                                                                                                        • The MIME type of the resource requested is included in the response, and it’s there where one can include a language tag (it’s even specified for text/gemini). At best, you can set a language per file, but if the document includes multiple languages, you are out of luck. But to be fair, does anyone actually tag foreign language words or phrases in HTML? I know I do (via tags) but I think I might be the only one.
                                                                                                                                                        • Images (like gif and jpegs) aren’t displayed inline. Yes, it’s an issue knowing a link is to an image (and what type of image) until it’s requested.
                                                                                                                                                        • The spec for text/gemini allow for “alt text” after the pre-formatted block marker, but there is no standard for how it should work, nor what it should contain. There’s been an insane amount of talk about the issue, but rarely (if ever) does someone even bother with a “proof-of-concept” to see how it might look or work (my biggest gripe with the Gemini community—mostly talk, no experimentation because actual working code is hard work and who wants to do that?)

                                                                                                                                                        Disclaimer: I wrote the first available Gemini server which helped with identifying the worst bits of the protocol (I did not bother much with text/gemini). I pretty much left the community because of the community, but I still run my Gemini site (gemini://gemini.conman.org/ in case anyone is interested).

                                                                                                                                                        1. 1
                                                                                                                                                          • A screen reader would know the language the same way anyone ever does. Language Detection is a pretty reasonably solved problem, certainly for long-ish passages.
                                                                                                                                                          • What images?
                                                                                                                                                          • That’s a screen reader problem (actually an ascii art problem), not a protocol accessibility problem. The web is no better at dealing with bad actors.

                                                                                                                                                          Feel free to write an Alexa client. It’d be pretty easy. (Like, legitimately so. It actually sounds kind of fun. I might try.)

                                                                                                                                                          1. 1

                                                                                                                                                            Those Figlet ASCII art images are absolute garbage for a screen reader

                                                                                                                                                            The standard allows for an optional “alt text” to be attached to preformatted sections.

                                                                                                                                                            Edit

                                                                                                                                                            How would a screen reader know what language a capsule is? What if you use multiple languages in your capsule?

                                                                                                                                                            The server I run (gemserv) has an optional directive indicating the language. I don’t know if it communicates this to the client though.

                                                                                                                                                2. 2

                                                                                                                                                  I’m not familiar with the insider politics of Gemini. Now I’m regretting wading into this. I’d have kept quiet if either blog post said, “the people on the mailing list are rude,” something I’m not qualified or interested to debate.

                                                                                                                                                  1. 2

                                                                                                                                                    it’s exclusionary for being mostly text based

                                                                                                                                                    Like the Amish.

                                                                                                                                                    If people want to live without electricity, who am I to tell them otherwise?

                                                                                                                                                    If I were ever interested in sharing math on Gemini it would be pretty much impossible;

                                                                                                                                                    Yes yes, all your calculus I’m sure is quite good, but there are other things too, and you might say nothing quite captures the beauty of a rose like a picture of a rose, and that music is best heard not talked about, and to say nothing of the medium of games and interactivity, where even being able to see all the code can spoil the ending!

                                                                                                                                                    We already have something perfectly mediocre at representing all of those things, but we don’t have anything really great at just doing text and links besides Gemini.

                                                                                                                                                    There’s nothing wrong with this, but it makes the community by definition exclusionary.

                                                                                                                                                    I disagree wholeheartedly: A meetup for blind people isn’t exclusionary if a sighted person can join. You are so welcome! You’re free to consume or produce whatever content you want, but so are they, and your inability to share your math with me speaks more of your abilities than mine for simply lacking the eyes with which to “read” it.

                                                                                                                                                    1. 14

                                                                                                                                                      Like the Amish.

                                                                                                                                                      The Amish don’t proselytize. In fact I suspect they’d be relieved if the outside world stopped being fascinated by them. This does not describe most Gemini evangelists.

                                                                                                                                                      We already have something perfectly mediocre at representing all of those things, but we don’t have anything really great at just doing text and links besides Gemini.

                                                                                                                                                      Well, views differ. I see the lack of semantic content for emphasis as crippling, for text. And I really cannot see any reason for this to be the case apart of the rigid “each line should be parseable as a unit” argument, and “it’s up to the client to decide how to present”.

                                                                                                                                                      Quoting myself from here:

                                                                                                                                                      italics and boldface are good, I am still mad

                                                                                                                                                      You can’t reproduce most prose works without having fugly underscores or asterisks all over. This is defacing our cultural heritage. Asking the user/client to download on the side and using an external reader is a cop-out.

                                                                                                                                                      It’s increasingly apparent to me that gemtext is well suited for one thing: writing about and discussing gemtext and Gemini. A bargain-basement Sapir-Worf theory, in other words.

                                                                                                                                                      Gemtext is designed by someone who thought that the pinnacle of human communication is a 1990s Usenet post. Gutenberg and Aldus Manutius wept.

                                                                                                                                                      1. 2

                                                                                                                                                        The Amish don’t proselytize.

                                                                                                                                                        I think that depends on what you mean by proselytize; Many Amish vote, and they have Amish lobbyists, and some are even quite mission-oriented in how they talk of their faith (e.g. NOA). Some have certain rules that you have to follow if you want to participate in their community (like for example, visiting an Amish church), and both (to me) seem quite tolerant of the existence of other faiths.

                                                                                                                                                        In any event, I don’t follow exactly is it about what Gemini “evangelism” that has anything to do with your inability to express yourself to blind people. To me, it’s like you’re telling people to remove the braille from elevators. Why? If you don’t want to use Gemini? Who is forcing you?

                                                                                                                                                        I see the lack of semantic content for emphasis as crippling. … I really cannot see any reason for this.

                                                                                                                                                        That’s too bad. Again, why do you care that something exists that isn’t for you? Do you bring your cat to dog-walking clubs as well? A child to the pub? Do you think everyone should like the things you like?

                                                                                                                                                        1. 3

                                                                                                                                                          I dunno man, I’m just here, working on my gemsite, participating on Antenna, hanging out in the #gemini IRC and generally having a good time. I’m sorry I’m not comporting myself befitting a member of the Church of Gemini.

                                                                                                                                                  2. 8

                                                                                                                                                    Twitter used to have a nice mobile site. It worked beautifully with text-mode browsers. I used to use it with edbrowse. They killed it in December 2019, a couple weeks after I quit twitter.

                                                                                                                                                    And then there’s twitter’s crusade against third-party clients. Third-party clients happen to be very popular with the blind.

                                                                                                                                                1. 24

                                                                                                                                                  I think Gemini is defined by what it isn’t rather than what it is (very punk rock), and that’ll be its ultimate downfall. Instead of trying to build something, they worry little nice touches will be the end and become the web again.

                                                                                                                                                  For a good example of what I mean, this is what I think of when I think of Gemini now - bullying people out of purity. (Or RFC LARP.)

                                                                                                                                                  1. 29

                                                                                                                                                    You’ve discovered that Drew Default is a dick. We all knew this. He’s been here. We know.

                                                                                                                                                    One could argue it’s a public service to provide a client that Drew Default will eventually blacklist. The users will be spared whatever his latest fucking insane windmill charge is. Think of the users.

                                                                                                                                                    1. 3

                                                                                                                                                      brave to attack someone in a forum where they cannot respond

                                                                                                                                                      1. 1

                                                                                                                                                        I sure hope Drew has better things to do than to respond to this kind of nonsense

                                                                                                                                                    2. 7

                                                                                                                                                      Or as Jawbreaker would have it:

                                                                                                                                                      “You’re not punk, and I’m telling everyone”

                                                                                                                                                      Save your breath, I never was one

                                                                                                                                                      Much like any clique, it’s got its purity tests. That’s a good example you’ve got there.

                                                                                                                                                      1. 5

                                                                                                                                                        It’s a curious perversion of open source development—who has the right to control or limit what other people do in their own software, especially adding features they find useful? One could argue that it’s daft, impractical or morally offensive, and they might be right. You can’t stop people publishing code, which leaves you with a rather primitive set tools—benevolent dictatorship, consensus and coercion—to keep a minimalist project on track. “Nice little touches” were never on the table, and favicons were only one in a litany of attempted extensions. Without continuous backpressure the markup definitely would have reached a level of complexity that browsing from a terminal would be undesirable. Most of the backpressure was achieved by solderpunk writing gently, but ultimately leaning heavily on their authority as the author of the spec to prevent unwanted excursions. Later they reduced their involvement so the availability of tools to keep things on track diminished.

                                                                                                                                                        That’s what Gemini is, for better or worse. Purity and ensuring purity is in its DNA. When I look at your link, I see that more as a reflection on the project than the person who wrote it. Even if others wouldn’t have been quite so blunt (or didn’t have the leverage to counter code with code) many were definitely thinking it. There’s a shock factor seeing that kind of thing on a GitHub issue—it would be a crazy demand in most software ecosystems, but in context I didn’t find it surprising.

                                                                                                                                                        1. 6

                                                                                                                                                          Without continuous backpressure the markup definitely would have reached a level of complexity that browsing from a terminal would be undesirable.

                                                                                                                                                          Browsing from a terminal isn’t one of the stated goals of the project though. I’m not sure if purity politics were essential for Gemini at the outset, but I do think that users of Gemini are interested in that kind of purity politics because of what Gemini and its purity represents for them.

                                                                                                                                                          1. 2

                                                                                                                                                            Browsing from a terminal isn’t one of the stated goals of the project though

                                                                                                                                                            Good point, I misremembered that this was more explicit. However it is a goal of gemtext that it should be usable without any processing of formatting, which amounts to much the same thing.

                                                                                                                                                            the text/gemini format has been designed so that simple clients can ignore the more advanced features and still remain very usable.

                                                                                                                                                            It is strictly optional for clients to do anything special at all with headings [/list items/blockquotes]

                                                                                                                                                            (gemtext format)

                                                                                                                                                            1. 4

                                                                                                                                                              Good point, I misremembered that this was more explicit. However it is a goal of gemtext that it should be usable without any processing of formatting, which amounts to much the same thing.

                                                                                                                                                              This exact split is reflected at large in the community and in practice ended with the minimalist group winning. For a lot of folks, they saw the goals of Gemini as a way to create a pure part of the Net where they can use their favorite tools (terminal, text editors, etc) to surf around. I contrast this with Usenet which has a much more laissez-faire attitude on what you can post in an article and how a newsreader should format an article.

                                                                                                                                                      1. 6

                                                                                                                                                        You might be better off trying to figure out why people aren’t upgrading from Python2.

                                                                                                                                                        1. 19

                                                                                                                                                          Oh, that’s easy, we’re still in the Python 2 hangover.

                                                                                                                                                          Godspeed to Python 3.6, IMO it was the first Python 3.x that had enough weight, new good features, and ported libraries to actually make a case to win people over from Python 2.x. And it literally just hit EOL.

                                                                                                                                                          I think the above comment as regards scripting vs applications is half of it, but the other half is that purely internal batch/data science/etc jobs and platforms also didn’t have a strong motivation to switch. If 2.7 was working and porting everything sucked (and it totally sucked, I did it for a team once) then why bother, especially if the benefits weren’t obvious.

                                                                                                                                                          And Python 3.6 was when it was obvious enough to actually take the time to do that work. Which, in a large company, takes forever to get everybody to start moving over, let alone completely actually move over. And it came out 5 years ago. Yes, it can take that long, especially when you consider it took a year or two just to get people into the idea that things were going to be okay now.

                                                                                                                                                          (edit: 5 years ago)

                                                                                                                                                          1. 5

                                                                                                                                                            Companies will keep using Python2 as long as some companies like RedHat are supporting it…

                                                                                                                                                            1. 3

                                                                                                                                                              Are redhat still supporting it? I don’t think it’s in rhel8. It is in rhel7 which is still in some kind of support but I’d have to check whether that was extended or not

                                                                                                                                                        1. 4

                                                                                                                                                          A very truncated version of the comment I have:

                                                                                                                                                          Phone screens are only on the surface level about code. Which is where you’re focusing. The human factor always gets in the way.

                                                                                                                                                          You’re trying to be a good coder. Try to be a charming coder.

                                                                                                                                                          Yes, the system is entirely bonkers.