1. 2

    I really want to like OCaml, but it has what I see as a fundamental misfeature, and I can’t tell whether I’m holding it wrong or whether there’s a good reason for it that I haven’t found yet. Specifically, partial application monomorphizes the result type.

    For example:

    # let id1 = fun a b -> b;;
    val id1 : 'a -> 'b -> 'b = <fun>
    # let id : 'a -> 'a = id1 "foo";;
    val id : '_a -> '_a = <fun>
    # id;;
    - : '_a -> '_a = <fun>
    # id 1;;
    - : int = 1
    # id;;
    - : int -> int = <fun>
    

    In my opinion, the type of id shouldn’t be monomorphized just because I happened to use it with a specific type. The fact that it does make it feel really awkward to define functions in a point-free style.

    What am I missing?

    1. 7

      Specifically, partial application monomorphizes the result type.

      Partial application only sometimes monomorphizes the result type. This ‘misfeature’ is caused by the relaxed value restriction, which is required to make the type system sound in the presence of mutation. You can read more about the value restriction in the OCaml manual.

      The fact that it does make it feel really awkward to define functions in a point-free style.

      In my experience, the value restriction is seldom an obstacle to most OCaml code. It’s not particularly common to mix point-free style pure code with code that uses side-effects and mutations—and when it happens, it’s probably a bad idea anyway, in terms of making it difficult to reason about the evaluation order. Plus, OCaml programmers don’t have as much of a desire to make their code point-free as, say, some Haskell folks seem to want to do.

      1. 4

        It’s not generally considered good practice to define functions in point-free style in OCaml, so there you have it.

        Whether this is due to the value restriction you mention or to the language’s general “aesthetics” is up for debate; personally I’ve never found the value restriction to be anything more than a mild annoyance.

      1. 12

        This is interesting, and I think I agree with many arguments when it comes to the reasons java, OCaml, Haskell, Go, etc. haven’t replaced C. However the author cites rust only briefly (and C++ not at all?) and doesn’t really convince me why rust isn’t the replacement he awaits: almost all you can do in C, you can do in rust (using “unsafe”, but that’s what rust is designed for after all), or in C++; you still have a higherer-level, safer (by default — can still write unsafe code where needed), unmanaged language that can speak the C ABI. Some projects have started replacing their C code with rust in an incremental (librsvg, I think? And of course firefox) because rust can speak the C ABI and use foreign memory like a good citizen of the systems world. Even C compilers are written in C++ these days.

        To me that’s more “some were meant for no-gc, C ABI speaking, unsafe-able languages” than “some were meant for C”. :-)

        1. 17

          Besides Rust, I think Zig, Nim, and D are strong contenders. Nothing against Rust, of course, but I’m not convinced it’s the best C replacement for every use case. It’s good to have options!

          Nonetheless, I imagine C will linger on for decades to come, just due to network effects and economics. Legacy codebases, especially low-level ones, often receive little maintenance effort relative to usage, and C code is incredibly widespread.

          1. 15

            I love Rust, but I think Zig and D (in the ‘better C’ mode and hopefully their new borrow checker) are closer to the simplicity and low-level functionality of C. Rust is a much nicer C++, with simpler (hah!) semantics and more room to improve. C++ is, unfortunately, a Frankenstein monster of a language that requires a 2000 page manual just to describe all the hidden weird things objects are doing behind your back. Every time I have to re-learn move semantics for a tiny project, I want to throw up.

            1. 3

              i was also wondering, while reading the article, how well ada would fit the author’s use case (i’m not at all familiar with the langauge, i’ve just heard it praised as a safe low-level language)

              1. 1

                The lot of them! It was kind of a large gap between C and Python/Perl/Ruby/Java.

                1. 12

                  Maybe I’m the archetype of a C-programmer not going for Rust. I appreciate Rust and as a Mathematician, I like the idea of hard guarantees that aren’t a given in C. However, Rust annoys me for three main reasons, and these are deal-breakers for me:

                  • Compile time: This is not merely the language’s fault, and has more to do with how LLVM is used, but there doesn’t seem to be much push to improve the situation, either. It annoys me as a developer, but it also really annoys me as a Gentoo user when a Firefox compilation takes longer and longer with each subsequent Rust release. Golang is a shining example for how you can actually improve compilation times over C. Admittedly, Rust has more static analysis, but damn is it slow to compile! I like efficiency, who doesn’t? Rust really drops the ball there.
                  • Standard library/external libraries: By trying to please everyone and not mandating certain solutions, one is constantly referred to this or that library on GitHub that is “usually used” and “recommended”. In other cases, there are two competing implementations. Sure, Rust is a young language, but for that reason alone I would never choose it to build anything serious on top of it, as one needs to be able to rely on interfaces. The Rust developers should stop trying to please everybody and come up with standard interfaces that also get shipped with the standard install.
                  • Cargo/Package management: This point is really close to the one before it: Cargo is an interesting system, but really ends up becoming a “monosolution” for Rust setups. Call me old-fashioned, but I like package managers (especially Gentoo’s) and Cargo just works around it. When installing a Rust package, you end up having to be connected to the internet and often end up downloading dozens of small crates from some shady GitHub repos. I won’t make the comparison with node.js, given Cargo can be “tagged” to a certain version, but I could imagine a similar scenario to leftpad in the future. Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.

                  To put it shortly: What I like about C is its simplicity and self-reliance. You don’t need Cargo to babysit it, you don’t need dozens of external crates to do basic stuff and it doesn’t get in the way of the package manager. I actually like Rust’s ownership system, but hate almost anything around it.

                  1. 16

                    C doesn’t even have a hash table. It needs more external libraries to do basic stuffs, not less.

                    1. 5

                      See, I feel the exact opposite when it comes to Cargo vs. system’s package manager: managing versions of libraries using your system’s package manager is a royal pain in the ass or outright impossible when you have multiple projects requiring different versions of a library. In my experience with C and C++, you’ll end up using CMake or Meson to build exactly the same functionality that Cargo deploys for you, at a much higher cost than just adding one line in a configuration file.

                      In fact, my biggest gripe with C and C++ is that they still depend on a 3-step build system (preprocessing, compiling, linking) each of which requires you to specify the location of a group of files. I get why having header files was attractive in the 1970s when you counted your computer’s memory in KBs, but it makes writing and maintaining code such a pain in the ass when compared with a modern module system.

                      The funniest bit is I used to consider all these things as ‘easy to deal with’ when all I did was write C/C++ code 15 years ago. Nowadays, having to switch from Go, Rust or any other language to C for a small project makes me want to cry because I know I’ll spend about 20% of the time managing bullshit that has nothing to do with the code I care about.

                      1. 9

                        Build systems in C give a feeling of craftsmanship. It takes a skill to write a Makefile that correctly supports parallel builds, exact dependencies, interruptions and cleanups, etc. And so much work into making it work across platforms, and compilers.

                        And then Cargo just makes it pointless. It’s like you were king’s best messenger trained to ride the fastest stallions, and Cargo’s like “thanks, but we’ve got e-mail”.

                        1. 2

                          LOL, I guess it’s a matter of age. When I first started programming, I’d love all that stuff. I can’t count how many libraries and tools I re-implemented or extended because they didn’t do something exactly the way I wanted it. Or the nights I spent configuring my Linux machine to work just right. Or the CPU time I spent re-encoding all of my MP3 collection to VBR because it’d save 5% of storage.

                          Now, I learned Cargo for a tiny project and I keep swearing every time I have to start a Python virtualenv because it’s just not easy enough, goddammit!.

                      2. 2

                        This is a fair criticism of C, personally I would love to see a set commonly used data structures added to the C standard library. However, currently in the C world you either write your own or use something like glib, neither of these cases require the equivalent of Cargo.

                        1. 4

                          However, currently in the C world you either write your own or use something like glib, neither of these cases require the equivalent of Cargo.

                          Neither does using the Rust standard library, which also has a hash table implementation (and many other useful data structures). You can just use std and compile your project with rustc.

                          1. 1

                            We’re talking about dependencies in general, not just hash tables. FRIGN’s point is that the Rust standard library is lacking, so you end up needing crates.

                            1. 3

                              But you and FRIGN are complaining about the Rust standard library compared to C. The Rust standard library is much more comprehensive than the C standard library or the C standard library + glib. So, the whole point seems to be void if C is the point of comparison.

                              If you are comparing to the Java standard library, sure!

                              1. 1

                                But you and FRIGN are complaining about the Rust standard library compared to C.

                                Not really. The point being made is that a typical Rust application has to download a bunch of stuff from Github (crates), where as a typical C application does not.

                                1. 8

                                  That’s just because it’s convenient and most people don’t really care that it happens. But it’s not inherent to the tooling:

                                  $ git clone -b ag/vendor-example https://github.com/BurntSushi/ripgrep
                                  $ cd ripgrep
                                  $ cargo build --release
                                  

                                  Other than the initial clone (obviously), nothing should be talking to GitHub or crates.io. You can even do cargo build --release --offline if you’re paranoid.

                                  I set that up in about 3 minutes. All I did was run cargo vendor, setup a .cargo/config to tell it to use the vendor directory, committed everything to a branch and pushed it. Easy peasy. If this were something a lot of people really cared about, you’d see this kind of setup more frequently. But people don’t really care as far as I can tell.

                                  where as a typical C application does not

                                  When was that last time you built a GNU C application? Last time I tried to build GNU grep, its build tooling downloaded a whole bunch of extra goop.

                                  1. -2

                                    Nice strawman, I said a typical C application, not a typical GNU C application.

                                    1. 5

                                      TIL that a GNU C application is not a “typical” C application. Lol.

                                      1. -1

                                        None of the C code I’ve worked on was written by GNU, and most of the C code out in the real world wasn’t written by GNU either. I find it frankly bizarre that you are seriously trying to suggest that GNU’s practices are somehow representative of all projects written in C.

                                        1. 3

                                          You said “a typical C application.” Now you’re saying “representative” and “what I’ve worked on.”

                                          If the implementation of coreutils for one of the most popular operating systems in history doesn’t constitute what’s “typical,” then I don’t know what does.

                                          Talk about bizarre.

                                          Moreover, you didn’t even bother to respond to the substance of my response, which was to point out that the tooling supports exactly what you want. People just don’t care. Instead, you’ve decided to double down on your own imprecise statement and have continued to shift the goal posts.

                                          1. 0

                                            and most of the C code out in the real world wasn’t written by GNU either.

                                            ^ typical

                                            I don’t have the time or patience to debate semantics though.

                                            As for your other point, see FRIGN’s comment for my response. (It doesn’t matter what’s possible when the reality is random crates get pulled from github repos)

                        2. 1

                          C doesn’t even have a hash table.

                          Why do you say “even”? There are many hash table implementations in C, with different compromises. It would be untoward if any of them made its way into the base language. There are other things missing in C which are arguably more fundamental (to me) before hash tables. It is only fair if all of these things are kept out of the language, lest the people whose favorite feature has not been included feel alienated by the changes.

                        3. 15

                          but there doesn’t seem to be much push to improve the situation

                          Definitely not true. There are people working on this and there has been quite a bit of progress:

                          $ git clone https://github.com/BurntSushi/ripgrep
                          $ cd ripgrep
                          $ git checkout 0.4.0
                          $ time cargo +1.12.0 build --release
                          
                          real    1:04.05
                          user    1:51.42
                          sys     2.282
                          maxmem  360 MB
                          faults  736
                          $ time cargo +1.43.1 build --release
                          
                          real    19.065
                          user    2:34.51
                          sys     3.101
                          maxmem  740 MB
                          faults  0
                          

                          That’s 30% of what it once was a few years ago. Pretty big improvement from my perspective. The compilation time improvements come from all around too. Whether it’s improving the efficiency of parallelism or micro-optimizing rustc itself: here, here, here, here, here, here or here.

                          People care.

                          The Rust developers should stop trying to please everybody and come up with standard interfaces that also get shipped with the standard install.

                          That’s one of std’s primary objectives. It has tons of interfaces in it.

                          This criticism is just so weird, given that your alternative is C. I mean, if you want the C experience of “simplicity and self-reliance,” then std alone is probably pretty close to sufficient. And if you want the full POSIX experience, bring in libc and code like its C. (Or maybe use a safe interface that somebody else has thoughtfully designed.)

                          When installing a Rust package, you end up having to be connected to the internet

                          You do not, at least, no more than you are with a normal Linux distro package manager. This was a hard requirement. Debian for example requires the ability to use Cargo without connecting to the Internet.

                          and often end up downloading dozens of small crates from some shady GitHub repos.

                          Yup, the way the crates.io model works means the burden of doing due diligence is placed on each person developing a Rust project. But if you’re fine with the spartan nature of C’s standard library, then you should be just fine using a pretty small set of well established crates that aren’t shady. Happy to see counter examples though!

                          but I could imagine a similar scenario to leftpad in the future.

                          The leftpad disaster was specifically caused by someone removing their package from the repository. You can’t do that with crates.io. You can “yank” crates, but they remain available. Yanking a crate just prevents new dependents from being published.

                          Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.

                          … like C? o_0

                          1. 10

                            This point is really close to the one before it: Cargo is an interesting system, but really ends up becoming a “monosolution” for Rust setups. Call me old-fashioned, but I like package managers (especially Gentoo’s) and Cargo just works around it.

                            C has just been in the luxurious position that its package managers have been the default system package managers. Most Linux package managers are effectively a C package managers. Of course, over time packages for other languages have been added, but they have mostly been second-class citizens.

                            It is logical that Cargo works around those package managers. Most of them are a mismatch for Rust/Go/node.js packages, because they are centered around distributing C libraries, headers, and binaries.

                            but I could imagine a similar scenario to leftpad in the future.

                            Rust et al. certainly have a much higher risk, since anyone can upload anything to crates.io. However, I think it is also an illusion that distribution maintainers are actually vetting code. In many cases maintainers will just bump versions and update hashes. Of course, there is some gatekeeping in that distributions usually only provide packages from better-known projects.

                            Rust really needs a better standard library so you don’t have to pull in so much stuff from other people.

                            You mean a large standard library like… C?

                            1. 3

                              C has just been in the luxurious position that its package managers have been the default system package managers.

                              This just isn’t true, if you look at how packages are built for Debian for example you will find that languages such as Python and Perl are just as well supported as C. No, the system package managers are for the most part language agnostic.

                              1. 5

                                This just isn’t true, if you look at how packages are built for Debian for example you will find that languages such as Python and Perl are just as well supported as C.

                                Most distributions only have a small subset of popular packages and usually only a limited number of versions (if multiple at all).

                                The fact that most Python development happens in virtual environments with pip-installed packages, even on personal machines, shows that most package managers and package sets are severely lacking for Python development.

                                s/Python/most non-C languages/

                                No, the system package managers are for the most part language agnostic.

                                Well if you define language agnostic as can dump files in a global namespace, because that’s typically enough for C libraries, sure. However, that does not work for many other languages, for various reasons, such as: no guaranteed ABI stability (so, any change down the chain of dependencies needs to trigger builds of all dependents, but there is no automated way to detect this, because packages are built in isolation), no strong tradition of ABI stability (various downstream users need different versions of a package), etc.

                                1. 5

                                  No, most development happens in virtualenv because python packaging is so broken that if you install a package you cannot reliably uninstall it.

                                  If we didn’t have a package manager for each language then the packages maintained by the OS would be more comprehensive, by necessity. Basically having a different packaging system for each programming language was a mistake in my view. I have some hope that Nix will remedy the situation somewhat.

                                  edit: it’s also difficult to reply to your comments if you substantially edit them by adding entirely new sections after posting…

                                  1. 5

                                    No, most development happens in virtualenv because python packaging is so broken that if you install a package you cannot reliably uninstall it.

                                    I have no idea what you mean here. Can’t you use dpkg/APT or rpm/DNF to uninstall a Python package?

                                    If we didn’t have a package manager for each language then the packages maintained by the OS would be more comprehensive, by necessity.

                                    We are going in circles. Why do you think languages have package managers? Technical reasons: the distribution package managers are too limited to handle what languages need. Social/political reasons: having the distributions as gatekeepers slows down the evolution of language ecosystems.

                                    I have some hope that Nix will remedy the situation somewhat.

                                    Nix (and Guix) can handle this, because it is powerful enough to implement the necessary language-specific packaging logic. In fact, Nix’ buildRustCrate is more or less an implementation of Cargo in Nix + shell script. It does not use Cargo. Moreover, Nix can handle a lot of the concerns that I mentioned upthread: it can easily handle multiple different versions of a package and ABI-instability. E.g. if in Nix the derivation of say the Rust compiler is updated, all packages of which Rust is a transitive dependency are rebuilt.

                                    As I said, traditional package managers are built for a C world. Not a Rust, Python, Go, or whatever world.

                            2. 4

                              The first problem is a technical one, unless Rust is doing things such that it can’t be compiled efficiently, but the latter two are cultural ones which point up differences in what language designers and implementers are expected to provide then versus now: In short, Rust tries to provide the total system, everything you need to build random Rust code you find online, whereas C doesn’t and never did. Rust is therefore in with JS, as you mention, but also Perl, Python, Ruby, and even Common Lisp now that Quicklisp and ASDF exist.

                              I was going to “blame” Perl and CPAN for this notion that language implementations should come with package management, but apparently CPAN was made in imitation of CTAN, the Comprehensive TeX Archive Network, so I guess this goes back even further. However, the blame isn’t with the language implementers at all: Packaging stuff is one of those things which has been re-invented so many times it’s bound to be re-invented a few more, simply because nobody can decide on a single way of doing it. Therefore, since language implementers can’t rely on OS package repos to have a rich selection up-to-date library versions, and rightly balk at the idea of making n different OS-specific packages for each version of each library, it’s only natural each language would reinvent that wheel. It makes even more sense when you consider people using old LTS OS releases, which won’t get newer library versions at this point, and consider longstanding practice from the days before OSes tended to have package management at all.

                              1. 7

                                Therefore, since language implementers can’t rely on OS package repos to have a rich selection up-to-date library versions, and rightly balk at the idea of making n different OS-specific packages for each version of each library, it’s only natural each language would reinvent that wheel.

                                This is right on the mark.

                                Sorry if this is a bit of a tangent, but I think it is not just a failing of package sets – from the distributor’s perspective it is impossible to package every Rust crate and rust crate version manually – but especially of package managers themselves. There is nothing that prevents a powerful package management system to generate package sets from Cargo.lock files. But most package managers were not built for generating package definitions programmatically and most package managers do not allow allow installing multiple package versions in parallel (e.g. ndarray 0.11.0 and ndarray 0.12.0).

                                Nix shows that this is definitely feasible, e.g. the combo of crate2nix and buildRustCrate can create Nix derivations for every dependency in a Cargo.lock file. It does not use cargo at all, compiles every crate into a separate Nix store path. As a result, Rust crates are not really different from any other package provided through nixpkgs.

                                I am not that familiar with Guix, but I bet it could do the same.

                              2. 3

                                Build times are important, and you’re right, Rust takes a while to compile. Given the choice between waiting for rustc to finish and spending a lot longer debugging a C program after the fact, I choose the former. Or, better yet, use D and get the best of both worlds.

                                1. 3

                                  rust can be damn fast to compile, most rust library authors just exercise fairly poor taste in my opinion and tend not to care how bad their build times get. sled.rs compiles in 6 seconds on my laptop, and most other embedded databases take a LOT longer (usually minutes), despite sled being Rust and them being C or C++.

                                  rust is a complex tool, and as such, you need to exercise judgement (which, admittedly, is rare, but that’s no different from anything else). you can avoid unnecessary genericism, proc macros, and trivial dependencies to get compile times that are extremely zippy.

                                  1. 2

                                    Thanks for your insights! I’ll keep it in mind the next time I try out Rust.

                                    1. 1

                                      Feel free to reach out if you hit any friction, I’m happy to point folks in the direction they want to go with Rust :)

                                2. 5

                                  almost all you can do in C, you can do in rust

                                  As an anecdata, I‘ve immediately recognized the snippet with elf header from the article, because I used one of the same tricks (just memcpying a repr(C) struct) for writing elf files in Rust a couple of months ago.

                                  1. 2

                                    I though about using rust to implement a byte-code compiler / vm for a gc’d language project, but I assumed that this would require too much fighting to escape rust’s ownership restrictions. Do you have any insight into how well suited rust is for vm implementation? I haven’t used the language much but I’d love to pick it up if I though I could make it work for my needs.

                                    (I see that there’s a python interpreter written in rust, but I’m having trouble locating its gc implementation)

                                    1. 5

                                      I honestly don’t know, I have never written an interpreter. You probably can fall back on unsafe for some things anyway, and still benefit from the move semantics, sum types, syntax, and friendly error messages. I’m doing a bit of exploring symbolic computations with rust and there are also some design space exploration to be done there.

                                      1. 2

                                        IMO, Rust is just as good as C and C++ for projects like this, if not better thanks to pattern matching and a focus on safety (which goes far beyond the borrow checker). Don’t be afraid to use raw pointers and NonNull pointers when they are appropriate.

                                        1. 2

                                          Also, just saw this GC for Rust on the front page: https://github.com/zesterer/broom/blob/master/README.md

                                          Looks like it’s designed specificly for writing dynamic languages in Rust.

                                          1. 1

                                            Oh cool! This looks super useful

                                        2. 2

                                          I wrote a ST80 VM in rust just to play around; the result was beautifully simple and didn’t require any unsafe code, though it doesn’t currently have any optimization at all. The result was still reasonably snappy, but I suspect that a big part of that is that the code I was running on it was designed in, well, 1980.

                                          1. 2

                                            I recently did a simple lisp interpreter in rust. Eventually decided to re-do it in c because of shared mutation and garbage collection.

                                            1. 2

                                              Rust works well for VM implementation. For GC, you may want to look at https://github.com/zesterer/broom.

                                          1. 24

                                            I’m gonna go with Qt on this one. I learned it a long time ago (I think it was still at version 2!) and it never really let me down. It’s got very good documentation, and it’s pretty reliable for long-term development. I have projects that have been through three Qt versions (3.x, 4.x, 5.x) and the migration has been pretty painless each time. It’s not on the nimble end of the spectrum and it’s C++, but I found it to be the most productive, even though the widgets library hasn’t been as high on the parent company’s priority list. (They insist that’s not true but actions speak louder than words…). I’ve used it for huge projects (200 KloC+) and it held out great.

                                            I used GTK 2 back in the day, too, and while some bits weren’t exactly enjoyable, it was generally efficient, and it was a pretty safe bet for cross-platform development, and an especially safe bet for Linux and Unix development. I really wanted to like GTK 3. I don’t know if it’s because I’m getting grumpy and impatient, or if there really is something objectively wrong with it, but I didn’t manage to like it, and now I tend to avoid it, both when it comes to writing code that uses it and when it comes to using applications written against it. Also I’m not sure how its cross-platformness is doing these days.

                                            I’ve played with Dear ImGui and I can definitely say I enjoy it. I’ve used it for some pretty small and special-purpose tools (and obviously you get about as much native integration with it as you get with Electron :P) but I definitely had fun with it. I’ve also

                                            1. 6

                                              I’m also a big fan of QT, and in particular, QtQuick is the single most productive rapid prototyping platform I’ve ever used (beating out even Visual Basic and Electron). The first app I ever wrote with it started out as an excuse to learn Qt Quick, and I had a working, polished app within two weeks.

                                              1. 4

                                                I really like Qt as well. I recently started building things with PyQt5 and it’s been pretty nice to work with:

                                                https://gitlab.com/djsumdog/mpvbuddy

                                                1. 2

                                                  +1 for Qt. I was surprised to see Telegram’s desktop client not using Electron, when every popular IM client is using it, and the UI seems much faster, and pleasant to work with. Another advantage is, Qt is available on more platforms than Electron, so if you like to be portable, don’t want to be limited by GNU/Linux, Windows, or macOS, then Qt is a good choice.

                                                  1. 1

                                                    I’ve also

                                                    Did you intend to continue?

                                                    1. 2

                                                      Did you intend to continue?

                                                      It looks like I did but whatever I wanted to say has long been swapped to the write-only section of my memory :)

                                                    2. 1

                                                      Happy with Qt too, but only when keeping the project up to date (and then it’s much easier with small projects). The least progress I’ve ever made as part of a software team was when we had a long-running Qt app where some parts were Qt5-ready, but we were mostly building with Qt4 and even then using 3-to-4 adapters in parts. Not that this isn’t true of other frameworks, but that sticks out as a raw nerve in my memory.

                                                      I’ve also used wxwidgets (but long enough ago that I don’t remember much specific, it seemed to work), GNUstep (OK if you don’t use any super-modern Cocoa APIs, where the approach to claiming 100% coverage has been to stub out all of the implementations), and Eclipse RCP which is a real curate’s egg.

                                                    1. 6

                                                      Smalltalk is an easy answer. :-)

                                                      In Smalltalk, you can replace the contents of any method in any class of the system. Obviously, you’d better not be wrong.

                                                      1. 2

                                                        Almost, but not quite: certain methods are actually opcodes (in ST80, arithmetic, ifTrue, and a couple of others), and modern smalltalks will open-code them, so overriding the method will do nothing unless the primitive fails. Even the ST80 blue book interpreter handles ifTrue directly in the bytecode, and the 32 “special” messages with their own opcodes result in directly calling a primitive without message lookup. Now, you’re not likely to want to override integer addition or Object>>class, but it still bothers me that you can’t.

                                                      1. 2

                                                        I’m curios when (and if) will see:

                                                        1. Enterprise offering/enforcing WG to their office workers
                                                        2. VPN “privacy” services offering WG
                                                        1. 4

                                                          Regarding #2, Mullvad already offers WG access.

                                                        1. 1

                                                          I don’t have the repos (“google3” and “fbcode”) available to me any more.

                                                          Are these the actual names of Google’s and Facebook’s monorepos?

                                                          1. 3

                                                            I can confirm that Google’s is google3; there was a google2 at least 15 years ago that used make as a build system, and I can only assume that there was a google at some point. No idea about facebook.

                                                            1. 1

                                                              An easy search online returned this: https://github.com/angular/google3

                                                              so I’d assume so.

                                                            1. 4

                                                              I really wish that people would stop recommending PEGs; in my experience, the cases where they do what you expect is exactly the set of cases when the underlying grammar is LL(1); if you have anything more complex, then the ordered choice construct will silently hide what the grammar author would want to be reported as an ambiguity and you’ll therefore end up parsing the wrong thing. Fixing this problem generally requires all sorts of hacks such as completeness or negative lookahead combinators (the latter of which are especially insidious as they need to be deep in the parser to avoid accepting too much).

                                                              This is not to say that the idea of parser combinators is bad; in fact, they have a wonderful API that has done far more to get people to use real parsers for small tasks. The problem is that there is a simple, intuitive implementation of parser combinators (i.e., PEGs) that will tend to behave in spectacularly unintuitive ways. It is possible (and in fact, relatively easy) to provide a parser combinator API for a LL(1), LALR, ALL*, or really any other parsing algorithm; see, for example hammer for how it might be done.

                                                              (Full disclosure: I wrote a significant portion of Hammer)

                                                              1. 1

                                                                Isn’t the parser combinator framework that this article talks about something distinct from a PEG? Am I misunderstanding what PEG refers to?

                                                                1. 1

                                                                  The class of grammar that this framework can parse is exactly PEG; this is easy to tell from the fact that the orElse combinator implements ordered choice. If the first match succeeds, the choice will never be re-examined even if it causes the grammar as a whole to fail.

                                                                  So, for example (with apologies for the formatting, as I don’t speak F#),

                                                                  sequence
                                                                    (orElse
                                                                      (many (pchar 'a'))
                                                                      (many (pchar 'b')))
                                                                    (pchar 'c')
                                                                  

                                                                  will fail on “bc”, because the first branch of the orElse succeeds having parsed nothing and then the parser for ‘c’ fails.

                                                                  Somebody who doesn’t know much about parsing would tend to imagine that that would be equivalent to the following:

                                                                  orElse
                                                                    (sequence (many (pchar 'a')) (pchar 'c'))
                                                                    (sequence (many (pchar 'b')) (pchar 'c'))
                                                                  

                                                                  (i.e., distributing the sequence into the orElse)

                                                                  However, this latter parser will accept “bc”

                                                                2. 1

                                                                  Any reason your changes aren’t upstreamed?

                                                                  1. 2

                                                                    UpstandingHackers is actually the primary repo; the reason it’s marked as a fork on Github is that my ex-wife didn’t want to deal with making it look right :-/

                                                                1. 18

                                                                  For folks wanting more context on how the “minimum supported Rust version” (MSRV) issue is treated in the ecosystem, this issue has a number of opinions (including my own) and some discussion: https://github.com/rust-lang/api-guidelines/issues/123

                                                                  As far as I can tell, there is no strong consensus on what to do. In practice, I’ve observed generally the following states:

                                                                  1. Some folks adopt an explicit MSRV policy but do not consider it a breaking change to increase it.
                                                                  2. Some folks adopt an explicit MSRV policy and consider it a breaking change to increase it.
                                                                  3. There is no MSRV policy, and the only guarantee you have is that it compiles on latest stable (or latest stable minus two releases).

                                                                  In general, I’ve found that (1) and (2) are usually associated with more widely used crates and generally indicates an overall more conservative approach to increasing the MSRV. (3) is generally the default though, as far as I can tell.

                                                                  There’s good reason for this. Maintaining support for older versions of Rust is a lot of thankless work, particularly if your library is still evolving or if your own crate has other dependencies with different MSRV policies. All it takes is one crate in your dependency graph to require a newer version of Rust. (Unless you’re willing to pin a dependency in a library, which is generally bad juju.) Rust’s release cycle reinforces this. It moves quickly and provides new things for folks to use all the time. Those new things are added specifically because folks have a use for them, so their use can propagate quickly in the ecosystem if a widely used crate starts using it. The general thinking here is that updating your Rust compiler should be easy. And generally speaking, it is.

                                                                  “Maturity” is perhaps the right word, but only in the sense that, over time, widely used crates will slow their pace of evolution and, consequently, slow their MSRV increases. This isn’t necessarily equivalent to saying that “maturity” equals “slow evolution,” because it is generally possible for crates to make use of newer versions of Rust without increasing their MSRV via version sniffing and conditional compilation. (Not possible in every case, but the vast majority.) But doing this can lead to significant complexity and a greatly increased test matrix. It’s a lot of extra work, and maybe doing that extra work is what this author means by “maturity.” Chances are though, that’s a lot of unpaid extra work, and it’s not clear to me that that is reasonable expectation to have.

                                                                  1. 4

                                                                    Perhaps part of the solution could be to make LTS versions of rustc and cargo? That way distro maintainers could preferentially use those, and package maintainers preferentially target those. Make the common toolchain setup procedure apt install cargo instead of curl https://sh.rustup.rs > sh and there’s at least a prayer of people preferring that. Debian 10 currently ships with rustc 1.34 for example, which IMO is a pretty good place to put a breakpoint.

                                                                    But for this to happen there needs to be agreement on what the LTS versions are. If Debian 10 ships rustc 1.34, Ubuntu 20.04 ships 1.37 and Fedora ships 1.12, then as a crate maintainer I’m not going to bother trying to target a useful minimal version, because it’s a lot of random work that will never be perfect. If everyone ships rustc 1.34, then it’s much easier to say to myself “well I’d like this shiny new feature in rustc 1.40 but I don’t really need it for now, it can just go in the next time I’m making a breaking release anyway”. This actually works in my favor, ‘cause then when a user tries to install my software on some ancient random system I can just say “sorry, you have to use rustc 1.34+ like everyone else, it’s not like that’s a big ask”. Then distro maintainers can backport rustc 1.34 to Debian 9 or 8 if they really need to, and only need to do it once as well for most people’s software to work.

                                                                    This happens already, hence why Debian 10 has gcc-7 and gcc-8 packages. It’s fine. The special cases just need to be uncommon enough that it’s not a huge hassle.

                                                                    1. 5

                                                                      Yes, people generally want some kind of LTS story. There was an RFC that was generally positively received about 1.5 years ago: https://github.com/rust-lang/rfcs/pull/2483

                                                                      It was closed due to lack of bandwidth to implement it, but it seems like something that will be revisited in the future. There’s just a ton of other stuff going on right now that is soaking up team bandwidth, mostly in the form of implementing already merged RFCs.

                                                                      1. 4

                                                                        It would be really sad to let Debian hold back Rust version adoption in the ecosystem the way Debian gets to hold back C++ version adoption via frozen GCC.

                                                                        It seems to me it would be a major strategic blunder for Rust to do an LTS instead of the current situation.

                                                                        1. 2

                                                                          Is Debian a factor anymore? I mean it was always pretty backwards, but does anybody use it, care for it anymore? How independent is Ubuntu from them?

                                                                          I only use fedora/centos/rhel or windows for work. I have only seen Ubuntu in use by others in large real-world deployments, but Debian? Never.

                                                                          1. 3

                                                                            Is Debian a factor anymore? I mean it was always pretty backwards, but does anybody use it, care for it anymore? How independent is Ubuntu from them?

                                                                            People do care about Debian and use Debian. That’s fine. What’s not fine is acting entitled to having code from outside the Debian stable archive build with the compilers shipped by Debian stable.

                                                                            As Ubuntu LTS releases get older, they have similar ecosystem problems as Debian stable generally, but in the case of Rust in particular, Ubuntu updates Rust on the non-ESR Firefox cycle, so Rust is exempt from being frozen in Ubuntu. (Squandering this exemption by doing a Rust LTS would be a huge blunder for Rust in my opinion.)

                                                                            In my anecdotal experience entitlement to have out-of-archive code build with in-archive compilers is less of a problem with RHEL. People seem to have a better understanding that if you use RHEL, you are paying Red Hat to deal with being frozen in time instead of being frozen in time being a community endeavor beyond the distro itself. Edited to add: Furthermore, in the case of Rust specifically, Red Hat provides a rolling toolchain for RHEL. It doesn’t roll every six weeks. IIRC, it updates about every third Rust upstream release.

                                                                            1. 3

                                                                              The company I work at (ISP & ISTP) use Debian as the operating system on almost all virtual machines running core software which requires n nines uptime.

                                                                              1. 3

                                                                                I’ve found Debian Stable to be perfectly fine for desktop and server use. It just works, and upgrades are generally pretty smooth. Clearly, you have different experiences, but that doesn’t make Debian “backwards”.

                                                                                1. 1

                                                                                  One department at my university is mostly-Debian for about 15+ years.

                                                                                  1. 0

                                                                                    I have seen Debian at a university department too, but not at places where actual money is made, or work is getting done. I had to use pkgsrc there to get fresh packages as a user to be able to get my stuff done.

                                                                                    University departments can afford to be backwards, because they are wasting other people’s time and money with that.

                                                                                    1. 3

                                                                                      Every place that I have worked primarily uses Debian or a Debian derivative. (Google used Ubuntu on workstations; at [Shiny consumer products, inc] the server that I was deploying on was Debian, despite the fact that they have their own server OS and they even supported it at the time; and the rest have been smaller firms or I’m under NDA and can’t discuss them). Except for Google, it was always Debian stable. So no, not just universities.

                                                                                      1. 1

                                                                                        BSD Unix was developed at a university.

                                                                                        Linus attended a university when starting to develop the Linux kernel.

                                                                                        The entire ethos and worldview of Free Software is inspired by RMS’ time at university.

                                                                                        The programming darling du jour, Haskell, is an offshoot of an academic project.

                                                                                        I’m really sad so much time and energy and other people’s money have been wasted on these useless things…

                                                                                        1. 2

                                                                                          Nice strawman!

                                                                                          And the infrastructure supporting these was just as backwards for its time as running Debian wasting the the time of students and tutors with outdated tools provided by the host institution…

                                                                                          1. 1

                                                                                            In the comment I replied to first , you write:

                                                                                            […] a university department too, but not at places where actual money is made, or work is getting done

                                                                                            University departments can afford to be backwards, because they are wasting other people’s time and money with that.

                                                                                            (my emphasis)

                                                                                            I find it hard to read these quotes in any other way than you believe that universities are a waste of time and money…

                                                                                            edit clarified source of quotes

                                                                                            1. 4

                                                                                              I can also mis-quote:

                                                                                              I find it hard to read […]

                                                                                              But I actually rather read and parse your sentences in their completeness.

                                                                                              My claims were:

                                                                                              a) I have only seen Debian used at places where efficiency is not a requirement
                                                                                              b) Universities are such places

                                                                                              I didn’t claim they don’t produce any useful things:

                                                                                              […] University departments can afford to be backwards, because they are wasting other people’s time and money with that.

                                                                                              Which should be parsed as: University Departments are wasting other people’s time and money with not using proper tools and infrastructure, for example using outdated (free) software. They are being inefficient. They waste student and tutor time, thus taxpayer money when not using better available free tools, but it doesn’t matter to them, as It does not show up n their balance sheet, Tutors and Students are already expected to do lot of “off-work hours” tasks to get their rewards: grades or money.

                                                                                              And yes, they are being inefficient:

                                                                                              • I had to find floppy disks in 2009 to be able to get my mandatory measurement data from a dos 5.0 machine at a lab. It was hard to buy them, and to get a place where I can read them… This one can be justified as expensive specialized measurement equipment was used and only legacy tools supported it.
                                                                                              • I had to do my assignments with software available only at the lab, running some Debian (then current) version shipping only outdated packages. OpenOffice kept crashing, and outdated tools were a constant annoyance. As a student my time was wasted. (Until I installed pkgsrc, and rolled my up to date tools)
                                                                                              • At a different university I have seen students working in Dosbox writing 16 bit protected mode in assembly in edit.com, compiling with some ancient MS assembler, in 2015, because the department thought the basics of assembly programming didn’t change since they introduced the curriculum, so they won’t update the tools and curriculum. They waste everyone’s money, the student’s won’t use it in real life anyway, because they are not properly supervised, as they would be if they were living from the market.
                                                                                              1. 3

                                                                                                Thanks for clarifying.

                                                                                                I realize it might be hard to realize for you now, but I can assure you that “the real world, governed by the market”, can be just as wasteful and inefficient as a university.

                                                                                                1. 2

                                                                                                  Unfortunately that is also true, I have seen “bullshit jobs” (a nice book btw.) business from inside (been partly a box- ticker for a time), but the enormous waste I saw at universities make me feel that the useful stuff coming out from them is exception, the result of herculean efforts of a few working against all odds, complete institutions working on strangling people/projects leading to meaningful results.

                                                                                                  Wasting someone’s own money is a thing, I don’t care that much about that, wasting taxpayer money is not a good move, but to some extent I can tolerate it… Wasting talents and other people’s time is what really infuriates me.

                                                                                        2. 1

                                                                                          I had to use pkgsrc there to get fresh packages

                                                                                          Did you have root privileges as a student?

                                                                                          1. 2

                                                                                            pkgsrc supports unprivilieged mode!

                                                                                            https://www.netbsd.org/docs/pkgsrc/platforms.html#bootstrapping-pkgsrc

                                                                                            It worked like a charm.

                                                                                            But I did actually have root privileges, as the guy responsible for the lab was overburdened and sometimes some of us he trusted helped other students. Still I didn’t use that to alter the installed system, as that would be out of my mandate.

                                                                                  2. 1

                                                                                    Debian 10 currently ships with rustc 1.34 for example, which IMO is a pretty good place to put a breakpoint.

                                                                                    1.34 doesn’t have futures nor async/await that seriously impact the code design. Do I really have to wait for Debian 11 in 2021 to use them?

                                                                                    1. 2

                                                                                      No, if you need them then install a newer rustc and use them. But there’s plenty of code that also doesn’t need futures or async/await.

                                                                                  3. 3

                                                                                    Wow, I wasn’t aware that this issue has an acronym and even a place for discussion. Thanks for the pointer!

                                                                                    widely used crates will slow their pace of evolution and, consequently, slow their MSRV increases.

                                                                                    Exactly what I’m hoping for, and precisely the reason I’m not jumping off the ship :)

                                                                                    maybe doing that extra work is what this author means by “maturity.”

                                                                                    In part, yes, that’s what I meant. The other possibility is to hold off adopting new APIs (as you did with alloc in regex; thanks!). I understand both options are a PITA for library maintainers, and might not even make sense, economy-wise, for unpaid maintainers. Perhaps I should’ve used “self-restraint” instead of “maturity”, but that probably has some unwanted connotations as well.

                                                                                    1. 2

                                                                                      Here’s a cargo subcommand (cargo msrv-table) I hacked together (warning, just a hacky PoC) that displays the MSRV by crate version for any particular crate.

                                                                                  1. 11

                                                                                    Whatever it’s faults there are plenty of good reasons to use protobuf not related to scale.

                                                                                    Protobuf is a well known, widely used format with support in many different programming languages. And it’s the default serialization format used by gRPC.

                                                                                    The only more boring choice would be restful JSON. But gRPC is actually easier to use for an internal service. You write a schema and get a client and server in lots of different languages.

                                                                                    And you also get access to an entire ecosystem of tools, like lyft’s envoy, automatic discovery, cli/graphical clients, etc.

                                                                                    Maybe instead of using an atypical serialization format (or god-forbid rolling your own), it would be better to spend your innovation tokens on something more interesting.

                                                                                    1. 11

                                                                                      The only more boring choice would be restful JSON.

                                                                                      ASN.1 anyone? For gods’ sake, it is one of the oldest protocol description format out there and for some reason people are still missing this out.

                                                                                      1. 10

                                                                                        I second ASN.1. Although underappreciated by modern tech stacks, it is used quite widely, e.g. in X.509, LDAP, SNMP and very extensively in telecom (SS7, GSM, GPRS, LTE, etc). It is suitable for protocols that need a unique encoding (distinguished encoding rules, DER) and for protocols where you can’t keep all the data in memory and need to stream it (BER).

                                                                                        It has some funny parts that might be better done away with, e.g. the numerous string types that nobody implements or cares about. I find it hilarious to have such things as TeletexString and VideotexString.

                                                                                        Support in today’s languages could be better. I suspect that Erlang has the best ASN.1 support of any language. The compiler, erlc, accepts ASN.1 modules straight up on the command line.

                                                                                        1. 11

                                                                                          If certs taught one thing to people it is that noone in their right mind should EVER use ASN.1 again.

                                                                                          1. 2

                                                                                            Nobody should use BER ever again, and people should use generated parsers rather than (badly) hand-rolling it. All of the certificate problems that I have seen are not fundamental to ASN.1, but rather badly hand-rolled implementations of BER.

                                                                                        2. 2

                                                                                          XDR/sunrpc predates even that by ~a decade I believe, and its tooling (rpcgen) is already available on most Linux systems without installing any special packages (it’s part of glibc).

                                                                                          1. 1

                                                                                            I love ASN.1 I guess other people prefer long text description of objects instead of a dot numbered notation.

                                                                                          2. 2

                                                                                            But gRPC is actually easier to use for an internal service. You write a schema and get a client and server in lots of different languages.

                                                                                            Swagger/openapi are so much better than grpc in that respect that it is borderline embarasing. No offense intended. It’s human readable and writeable. You can include as much detail as you want. For example you can include only the method signatures or you can include all sorts of validation rules. You can include docstrings. You have an interactive test GUI out of the box which you don’t need to distribute. All they need is the url for you swagger spec. There are tools to generate client libraries for whatever languages you fancy, certainly more than those grpc offers, in some cases multiple library generators per language. But most importantly. It doesn’t force you do distribute anything. There is no compile step necessary. Simply call the API via http, you can even forge your requests by hand.

                                                                                            In a job I had, we replaced a couple of HTTP APIs with gRPC because a couple of Google fanboys thought it was critical to spend time fixing something that just works with whatever Google claims to be the be all end all solution. The maintaining effort for those APIs jumped up an order of magnitude easily.

                                                                                            1. 2

                                                                                              gRPC with protobuf is significantly simpler than a full-blown HTTP API. In this regard gRPC is less flexible, but if you don’t need those features (ie you really are just building an RPC service), it’s a lot easier to write and maintain. (I’ve found swagger to be a bit overwhelming every time I’ve looked at it)

                                                                                              Why was there so much maintenance effort for gRPC? Adding a property or method is a single line of code and you just regenerate the client/server code. Maybe the issue was familiarity with the tooling? gRPC is quite well documented and there are plenty of stack-overflowable answers to questions.

                                                                                              I’ve only ever used gRPC with python and Go. The python library had some issues, but most of them were fixed over time. Maybe you were using a language that didn’t play nice?

                                                                                              Also this has nothing to do with Google fanboyism. I worked at a company where we used the Redis protocol for our RPC layer, and it had significant limitations. In our case, there was no easy way to transfer metadata along with a request. We need the ability to pass through a trace ID and we also wanted support for cancellation and timeouts. You get all that out of the box with gRPC. (in Go you use context) We looked at other alternatives and there were either missing features we wanted or the choice was so esoteric that we were afraid it would present too much of an upfront hurdle for incoming developers.

                                                                                              I guess we could’ve gone with thrift. But gRPC seemed easier to use.

                                                                                            2. 1

                                                                                              And it’s the default serialization format used by gRPC.

                                                                                              We got stuck using protobufs at work, and they’ve been universally reviled by our team as being a pain in the neck to work with, merely because of their close association with gRPC. I don’t think the people making the decision realized that gRPC could have the encoding mechanism swapped out. Eventually we switched to a better encoding, but it was a long, tedious road to get there.

                                                                                              1. 1

                                                                                                What problems have you had with protobufs? All the problems the original post talks about come from the tool evolving over time while trying to maintain as much format compatibility as possible. While I agree the result is kind of messy, I’ve never seen any of those quirks actually cause significant problems in practice.

                                                                                                The two biggest complaints I’ve heard about gRPC are “Go gRPC is buggy” and “I’m annoyed I can’t serialize random crap without changing the proto schema.” Based on what I know about your personal projects, I can’t imagine you having either problem.

                                                                                                1. 2

                                                                                                  Part of the problem is that the Java API is very tedious to use from Clojure, and part of the problem is that you inherit certain properties of golang’s pants-on-head-stupid type system into the JVM, like having nils get converted into zeroes or the empty string. Having no way to represent UUIDs or Instants caused a lot of tedious conversion. And like golang, you can forget about parametric types.

                                                                                                  (This was in a system where the performance implications of the encoding were completely irrelevant; much bigger bottlenecks were present several other places in the pipeline, so optimizing at the expense of maintainability made no sense.)

                                                                                                  But it’s also just super annoying because we use Clojure spec to describe the shape of our data in every other context, which is dramatically more expressive, plus it has excellent tooling for test mocks, and allows us to write generative tests. Eventually we used Spec alongside Protobufs, but early on the people who built the system thought it would be “good enough” to skip Spec because Protobufs “already gives us types”, and that was a big mistake.

                                                                                                  1. 1

                                                                                                    Thanks for the detailed reply! I can definitely see how Clojure and protobufs don’t work well together. Even without spec, the natural way to represent data in Clojure just doesn’t line up with protobufs.

                                                                                            1. 2

                                                                                              Something like this has happened before. In the early days of the ARPANET, you could fairly easily get access to one of the MIT ITS machines, which was (I’m given to understand) very much a communal environment. There were mechanisms to see what other people were doing on the system, to look over the shoulder of somebody attached to a network terminal, to ask for help, etc. There is also SDF, which (still!) provides a shared UNIX system with accounts available to anybody who asks, and in fact started out as a bulletin board system.

                                                                                              Both of these facilities were (and in SDF’s case, still are) inherently social. As I understand it, though (keep in mind, this was all before my time), the dream was always for computing to be personal, to have the complete resources of an entire machine at one person’s command. The existence of the shared access systems was a side effect of the fact that computers were too expensive for a single person to own.

                                                                                              I don’t actually have a point here, just more to reflect on :-D

                                                                                              1. 1

                                                                                                The ultimate in retro challenge: Install ITS and actually make it do something. Prepare to learn PDP-10 assembler :)

                                                                                                1. 2

                                                                                                  Much easier these days; there’s a script at https://github.com/PDP-10/its that will do everything for you. Good luck changing your hostname, though; that does still require PDP-10 assembly.

                                                                                                1. 7

                                                                                                  Well, some of us are in this category (as the article points out):

                                                                                                  If you’re building API services that need to support server-to-server or client-to-server (like a mobile app or single page app (SPA)) communication, using JWTs as your API tokens is a very smart idea. In this scenario:

                                                                                                  • You will have an authentication API which clients authenticate against, and get back a JWT
                                                                                                  • Clients then use this JWT to send authenticated requests to other API services These other API services use the client’s JWT to validate the client is trusted and can perform some action without needing to perform a network validation

                                                                                                  so JWT is not that bad. Plus, it is refreshing to visit a website that says ‘there are no cookies here’… in their privacy policy.

                                                                                                  1. 17

                                                                                                    Plus, it is refreshing to visit a website that says ‘there are no cookies here’… in their privacy policy.

                                                                                                    The EU “Cookie Law” applies to all methods of identification — cookies, local storage, JWT, parameters in the URL, even canvas fingerprinting. So it shouldn’t have any effect on the privacy policy whatsoever.

                                                                                                    1. 9

                                                                                                      You still can use sessions with cookies, especially with SPA. Unless the JWT token is stateless and short lived you should not use it. Also JWT isn’t the best design either as it gives too much flexibility and too much possibilities to misuse. PASETO tries to resolve these problems with versioning protocol and reducing amount of possible hashes/encryption methods.

                                                                                                      1. 1

                                                                                                        Why shouldn’t you use long lived JWTs with a single page application?

                                                                                                        1. 4

                                                                                                          Because you cannot invalidate that token.

                                                                                                          1. 6

                                                                                                            Putting my pedant hat on: technically you can, using blacklists or swapping signing files; But that then negates the benefit of encapsulating a user “auth key” into a token because the server will have to do a database lookup anyway and by that point might as well be a traditional cookie backed session.

                                                                                                            JWTs are useful when short lived for “server-less”/lambda api’s so they can authenticate the request and move along quickly but for more traditional things they can present more challenges than solutions.

                                                                                                            1. 7

                                                                                                              Putting my pedant hat on: technically you can, using blacklists or swapping signing files; But that then negates the benefit of encapsulating a user “auth key” into a token because the server will have to do a database lookup anyway and by that point might as well be a traditional cookie backed session.

                                                                                                              Yes, that was my point. It was just mental shortcut, that if you do that, then there is no difference between “good ol’” sessions and using JWT.

                                                                                                              Simple flow chart.

                                                                                                              1. 1

                                                                                                                Except it is not exactly the same since loosing a blacklist database is not the same as loosing a token database for instance. The former will not invalidate all sessions but will re-enabled old tokens. Which may not be that bad if the tokens are sufficiently short-lived.

                                                                                                                1. 1

                                                                                                                  Except “reissuing” old tokens has much less impact (at most your clients will be a little annoyed) than allowing leaked tokens to be valid again. If I would be a client I would much more like the former rather than later.

                                                                                                      2. 5

                                                                                                        One of my major concerns with JWT’s is that retraction is a problem.

                                                                                                        Suppose that I have the requirement that old authenticated sessions have to be remotely retractable, then how on earth would I make a certain JWT invalid without having to consult the database for “expired sessions”.

                                                                                                        The JWT to be invalidated could still reside on the devices of certain users after it has been invalidated remotely.

                                                                                                        The only way I could think of, is making them so short-lived that they expire almost instantaneous. Like in a few minutes at most, which means that user-sessions will be terminated annoyingly fast as well.

                                                                                                        If I can get nearly infinite sessions and instant retractions, I will gladly pay the price of hitting the database on each request.

                                                                                                        1. 8

                                                                                                          JWT retraction can be handled in the same way that a traditional API token would; you add it to a black list, or in the case of a JWT a “secret” that its signed against can be changed. However both solutions negate the advertised benefit of JWTs or rather they negate the benefits I have seen JWTs advertised for: namely that it removes the need for session lookup on database.

                                                                                                          I have used short lived JWTs for communicating with various stateless (server-less/lambda) api’s and for that purpose they work quite well; each endpoint has a certificate they can check the JWT validity with and having the users profile and permissions encapsulated means not needing a database connection to know what the user is allowed to do; a 60s validity period gives the request enough time to authenticate before the token expires while removing the need for retraction.

                                                                                                          I think the problem with JWTs is that many people have attempted to use them as a solution for a problem already better solved by other things that have been around and battle tested for much longer.

                                                                                                          1. 7

                                                                                                            However both solutions negate the advertised benefit of JWTs or rather they negate the benefits I have seen JWTs advertised for: namely that it removes the need for session lookup on database.

                                                                                                            I think the problem with JWTs is that many people have attempted to use them as a solution for a problem already better solved by other things that have been around and battle tested for much longer.

                                                                                                            This is exactly my main concern and also the single reason I haven’t used JWT’s anywhere yet. I can imagine services where JWT’s would be useful, but I have yet to see or build one where some form of retraction wasn’t a requirement.

                                                                                                            My usual go-to solution is to generate some 50-100 characters long string of gibberish and store that into a cookie on the user’s machine and a database table consisting of <user_uuid, token_string, expiration_timestamp> triples which is then joined with the table which contains user-data. Such queries are usually blazing fast and retraction then is a simple DELETE-query. Also: Scaling usually isn’t that big of a concern as most DBMS-systems tend to have the required features built-in already.

                                                                                                            Usually, I also set up some scheduled event in the DMBS which deletes all expired tokens from that table periodically. Typically once per day at night, or when the amount of active users is low. It makes for a nice fallback just in case some programming bug inadvertently creeps in.

                                                                                                            But I guess this was the original author’s point as well.

                                                                                                          2. 1

                                                                                                            I’ve never done any work with JWTs so this might be a dumb question - but can’t you just put an expiration time into the JWT data itself, along with the session and/or user information? The user can’t alter the expiration time because presumably that would invalidate the signature, so as long as the timestamp is less than $(current_time) you’d be good to go? I’m sure I’m missing something obvious.

                                                                                                            1. 5

                                                                                                              If someone steals the JWT they have free reign until it expires. With a session, you can remotely revoke it.

                                                                                                              1. 1

                                                                                                                That’s not true. You just put a black mark next to it and every request after that will be denied - and it won’t be refreshed. Then you delete it once it expires.

                                                                                                                1. 7

                                                                                                                  That’s not true. You just put a black mark next to it and every request after that will be denied - and it won’t be refreshed. Then you delete it once it expires.

                                                                                                                  The problem with the black mark, is that you have to hit some sort of database to check for that black mark. By doing so, you invalidate the usefulness of JWT’s. That is one of OP’s main points.

                                                                                                                  1. 2

                                                                                                                    Well, not necessarily. If you’re making requests often (e.g, every couple of seconds) and you can live with a short delay between logging out and the session being invalidated, you can set the timeout on the JWT to be ~30 seconds or so and only check the blacklist if the JWT is expired (and, if the session isn’t blacklisted, issue a new JWT). This can save a significant number of database requests for a chatty API (like you might find in a chat protocol).

                                                                                                                    1. 1

                                                                                                                      Or refresh a local cache of the blacklist periodically on each server, so it’s a purely in-memory lookup.

                                                                                                                      1. 4

                                                                                                                        But in that case, you’d be defeating their use as session tokens, because you are limited to very short sessions. You are just one hiccup of the network away from failure which also defeats their purpose. (which was another point of the OP).

                                                                                                                        I see how they can be useful in situations where you are making a lot of requests, but the point is that 99,9% of websites don’t do that.

                                                                                                            2. 1

                                                                                                              For mobile apps, that have safe storage for passwords, the retraction problem is solved via issuing refresh tokens (that live longer, like passwords in password store of a mobile phone). The refresh tokens, are then used to issue new authorization token periodically and it is transparent to the user. You can re issue authorization token, using refresh token every 15 minutes, for example.

                                                                                                              For web browsers, using refresh tokens may or may not be a good idea. Refresh tokens, are, from the security prospective, same as ‘passwords’ (although temporary). So their storage within web browser, should follow same policy as one would have for passwords.

                                                                                                              So if using refresh tokens for your single page app, is not an option, then invalidating would have to happen during access control validation, on the backend. (Backend, still is responsible for access control, anyway, because it cannot be done on web clients, securely).

                                                                                                              It is more expensive, and requires a form of distributed cache if you have distributed backend that allows stateless no-ip-bound distribution of requests…

                                                                                                              1. 1

                                                                                                                For mobile apps, that have safe storage for passwords, the retraction problem is solved via issuing refresh tokens (that live longer, like passwords in password store of a mobile phone).

                                                                                                                But then why use 2 tokens instead of single one? It makes everything more complicated for sake of perceived simplification of not doing 1 DB request on each connection. Meh. And even you can use cookie as in your web UI, so in the end it will make everything simpler as you do not need to use 2 separate auth systems in your app.

                                                                                                                1. 1

                                                                                                                  It makes everything more complicated for sake of perceived simplification of not doing 1 DB request on each connection.

                                                                                                                  This is not really, why 2 tokens are used (authentication token, and refresh token). 2 tokens are used to a) allow fast expiration of an authentication request b) prevent passing of actual user password through to the backend (it only needs to be passed when creating a refresh token).

                                                                                                                  This is a fairly standard practice though, not something I invented (it requires an API accessible, secure password store on user’s device ,which is why it is prevalent in mobile apps).

                                                                                                                  I also cannot see how a) and b) can be achieved with a single token.

                                                                                                          1. 33

                                                                                                            My position has essentially boiled down to “YAML is the worst config file format, except for all the other ones.”

                                                                                                            It gets pretty bad if your documents are large or if you need to collaborate (it’s possible to have a pretty good understanding of parts of YAML but that’s not always going to line up with what your collaborators understand).

                                                                                                            I keep wanting to say something along the lines of “oh, YAML is fine as long as you stick to a reasonable subset of it and avoid confusing constructs,” but I strongly believe that memory-unsafe languages like C/C++ should be abandoned for the same reason.

                                                                                                            JSON is unusable (no comments, easy to make mistakes) as a config file format. XML is incredibly annoying to read or write. TOML is much more complex than it appears… I wonder if the situation will improve at any point.

                                                                                                            1. 23

                                                                                                              I think TOML is better than YAML. Sure, it has the complex date stuff, but that has never caused big surprises for me (just small annoyances). The article seems to focus mostly on how TOML is not Python, which it indeed is not.

                                                                                                              1. 14

                                                                                                                It’s syntactically noisy.

                                                                                                                Human language is also syntactically noisy. It evolved that way for a reason: you can still recover the meaning even if some of the message was lost to inattention.

                                                                                                                I have a mixed feeling about TOML’s tables syntax. I would rather have explicit delimiters like curly braces. But, if the goal is to keep INI-like syntax, then it’s probably the best thing to do. The things I find really annoying is inline tables.

                                                                                                                As of user-typed values, I came to conclusion that everything that isn’t an array or a hash should just be treated as a string. If you take user input, you cannot just assume that the type is correct and need to check or convert it anyway, so why even bother having different types at the format level?

                                                                                                                Regardless, my experience with TOML has been better than with alternatives, despite its flaws.

                                                                                                                1. 6

                                                                                                                  Human language is also syntactically noisy. It evolved that way for a reason: you can still recover the meaning even if some of the message was lost to inattention.

                                                                                                                  I have a mixed feeling about TOML’s tables syntax. I would rather have explicit delimiters like curly braces. But, if the goal is to keep INI-like syntax, then it’s probably the best thing to do. The things I find really annoying is inline tables.

                                                                                                                  It’s funny how the exact same ideas made me make the opposite decision. I came to the conclusion that “the pain has to be felt somewhere” and that the config files are not the worst place to feel it.

                                                                                                                  I have mostly given up on different config formats and just default to one of the following three options:

                                                                                                                  1. Write .ini or Java properties-file style config-files when I don’t need more.
                                                                                                                  2. Write a dtd and XML when I need tree or dependency-like structures.
                                                                                                                  3. Store the configuration in a few tables inside an RDBMS and drop an .ini-style config file with just connection settings and the name of the config tables when things get complex.

                                                                                                                  As of user-typed values, I came to conclusion that everything that isn’t an array or a hash should just be treated as a string. If you take user input, you cannot just assume that the type is correct and need to check or convert it anyway, so why even bother having different types at the format level?

                                                                                                                  I fully agree with this well.

                                                                                                                2. 23

                                                                                                                  Dhall is looking really good! Some highlights from the website:

                                                                                                                  • Dhall is a programmable configuration language that you can think of as: JSON + functions + types + imports
                                                                                                                  • You can also automatically remove all indirection in any Dhall code, converting the file to a logic-free normal form for non-programmers to understand.
                                                                                                                  • We take language security seriously so that your Dhall programs never fail, hang, crash, leak secrets, or compromise your system.
                                                                                                                  • The language aims to support safely importing and evaluating untrusted Dhall code, even code authored by malicious users.
                                                                                                                  • You can convert both ways between Dhall and JSON/YAML or read Dhall configuration files directly into a language that supports a native language binding.
                                                                                                                  1. 8

                                                                                                                    I don’t think the tooling should be underestimated, too. The dhall executable includes low-level plumbing tools (individual type checking, importing, normalization), a REPL, a code formatter, a code linter to help with language upgrades, and there’s full blown LSP integration. I enjoy writing Dhall so much that for new projects I’m taking a more traditional split between a core “engine”, and then pushing out the logic into Dhall - then compiling it at a load time into something the engine can work with. The last piece of the puzzle to me is probably bidirectional type inference.

                                                                                                                    1. 2

                                                                                                                      That looks beautiful! Can’t wait to give it a go on some future projects.

                                                                                                                      1. 2

                                                                                                                        Although the feature set is extensive, is it really necessary to have such complex functionality in a configuration language?

                                                                                                                        1. 4

                                                                                                                          It’s worth understanding what the complexity is. The abbreviated feature set is:

                                                                                                                          • Static types
                                                                                                                          • First class importing
                                                                                                                          • Function abstraction

                                                                                                                          Once I view it through this light, I find it easier to convince myself that these are necessary features.

                                                                                                                          • Static types enforce a schema on configuration files. There is almost always a schema on configuration, as something is ultimately trying to pull information out of it. Having this schema reified into types means that other tooling can make use of the schema - e.g., the VS Code LSP can give me feedback as I edit configuration files to make sure they are valid. I can also do validation in my CI to make sure my config is actually going to be accepted at runtime. This is all a win.

                                                                                                                          • Importing means that I’m not restricted to a single file. This gives me the advantage of being able to separate a configuration file into smaller files, which can help decompose a problem. It also means I can re-use bits of configuration without duplication - for example, maybe staging and production share a common configuration stanza - I can now factor that out into a separate file.

                                                                                                                          • Function abstraction gives me a way to keep my configuration DRY. For example, if I’m configuring nginx and multiple virtual hosts all need the same proxy settings, I can write that once, and abstract out my intention with a function that builds a virtual host. This avoids configuration drift, where one part is left stale and the rest of the configuration drifts away.

                                                                                                                          1. 1

                                                                                                                            That’s very interesting, I hadn’t thought of it like that. Do you mostly use Dhall itself as configuration file or do you use it to generate json/yaml configuration files?

                                                                                                                        2. 1

                                                                                                                          I finally need to implement Dhall evaluator in Erlang for my projects. I <3 ideas behind Dhall.

                                                                                                                        3. 5

                                                                                                                          I am not sure that there aren’t better options. I am probably biased as I work at Google, but I find Protocol Buffer syntax to be perfectly good, and the enforced schema is very handy. I work with Kubernetes as part of my job, and I regularly screw up the YAML or don’t really know what the YAML is so cutty-pasty from tutorials without actually understanding.

                                                                                                                          1. 4

                                                                                                                            Using protobuf for config files sounds like a really strange idea, but I can’t find any arguments against it.
                                                                                                                            If it’s considered normal to use a serialisation format as human-readable config (XML, JSON, S-expressions etc), surely protobuf is fair game. (The idea of “compiled vs interpreted config file” is amusing though.)

                                                                                                                            1. 3

                                                                                                                              I have experience with using protobuf to communicate configuration-like information between processes and the schema that specifies the configurations, including (nested) structs/hashes and arrays, ended up really hacky. I forgot the details, but protobuf lacks one or more essential ingredients to nicely specify what we wanted it to specify. As soon as you give up and allow more dynamic messages, you’re of course back to having to check everything using custom code on both sides. If you do that, you may as well just go back to yaml. The enforced schema and multi language support makes it very convenient, but it’s no picnic.

                                                                                                                              1. 2

                                                                                                                                One issue here is that knowing how to interpret the config file’s bytes depends on having the protobuf definition it corresponds to available. (One could argue the same is true of any config file and what interprets it, but with human-readable formats it’s generally easier to glean the intention than with a packed binary structure.)

                                                                                                                                1. 2

                                                                                                                                  At Google, at least 10 years ago, the protobuf text format was widely used as a config format. The binary format less so (but still done in some circumstances when the config file wouldn’t be modified by a person).

                                                                                                                                  1. 3

                                                                                                                                    TIL protobuf even had a text format. It sounds like it’s not interoperable between implementations/isn’t “fully portable”, and that proto3 has a JSON format that’s preferable.. but then we’re back to JSON.

                                                                                                                            2. 2

                                                                                                                              JSON can be validated with a schema (lots of tools support it, including VSCode), and it’s possible to insert comments in unused fields of the object, e.g. comment or $comment.

                                                                                                                              1. 17

                                                                                                                                and it’s possible to insert comments in unused fields of the object, e.g. comment or $comment.

                                                                                                                                I don’t like how this is essentially a hack, and not something designed into the spec.

                                                                                                                                1. 2

                                                                                                                                  Those same tools (and often the system on the other end ingesting the configuration) often reject unknown fields, so this comment hack doesn’t really work.

                                                                                                                                  1. 8

                                                                                                                                    And not without good reason: if you don’t reject unknown fields it can be pretty difficult to catch misspellings of optional field names.

                                                                                                                                    1. 2

                                                                                                                                      I’ve also seen it harder to add new fields without rejecting unknown fields: you don’t know who’s using that field name for their own use and sending it to you (intentionally or otherwise).

                                                                                                                                  2. 1

                                                                                                                                    Yes, JSON can be validated by schema. But in my experience, JSON schema implementations are widely diverging and it’s easy to write schemas that just work in your particular parser.

                                                                                                                                  3. 1

                                                                                                                                    JSON is unusable (no comments, easy to make mistakes) as a config file format.

                                                                                                                                    JSON5 fixes this problem without falling prey to the issues in the article: https://json5.org/

                                                                                                                                    1. 2

                                                                                                                                      Yeah, and then you lose the main advantage of json, which is how ubiquitous it is.

                                                                                                                                      1. 1

                                                                                                                                        In the context of a config format, this isn’t really an advantage, because only one piece of code will ever be parsing it. But this could be true in other contexts.

                                                                                                                                        I typically find that in the places where YAML has been chosen over JSON, it’s usually for config formats where the ability to comment is crucial.

                                                                                                                                  1. 4

                                                                                                                                    If only there were a standardized binary format with JSON semantics and a canonical form.

                                                                                                                                    1. 5

                                                                                                                                      There’s CBOR (Concise Binary Object Representation), where the RFC does mention a canonical form.

                                                                                                                                      1. 2

                                                                                                                                        Unfortunately, like the rest of CBOR in my experience, the canonical form is neither as simple nor as rigorous as one might desire.

                                                                                                                                        1. 1

                                                                                                                                          Would you elaborate, please? I was under the impression that strict+canonical form following the recommendations in the RFC and normalizing unicode strings was safe.

                                                                                                                                          Do you have any counter example?

                                                                                                                                          1. 1

                                                                                                                                            The recommendations are recommendations, so a conforming implementation does not need to follow them. Following them is actually not trivial, given that it involves things like selecting sizes for integers that are far more grainular than I care about, ie, integers other than i32 or i64. So my implementation needs to be able to emit more complicated CBOR than it would otherwise find necessary. It says “integers must be as small as possible”, but is that as small as the value, or as small as the value’s possible domain? It’s unclear. It provides essentially no canonicalization for floats, just another suggestion or two. And normalizing Unicode strings, as the original post points out, is far from easy, though I suppose you’re going to have to do it somewhere no matter what.

                                                                                                                                            It’s not a canonical format, it’s a mixed bag of suggestions and directives for how to start designing a canonical format. It’s possible for two canonical parsers/emitters disagree on the same data, with details such as float signaling bits, whether or not duplicates are okay, etc.

                                                                                                                                            …actually, now that I look at it, CBOR in general is very weird. It describes very precisely what the representation of a data item should look like based off what the data is, and not what the the data may be. Maybe I’ve been programming strongly typed languages too long but that’s almost never what I actually want. Hm, I should try looking at it again from the opposite angle.

                                                                                                                                        2. 2

                                                                                                                                          Exactly. And it’s apparently still not widely known. It also supports chunked transmission if anyone needed to stream some JSON-like data and needed a framing protocol.

                                                                                                                                        3. 4

                                                                                                                                          ASN.1?

                                                                                                                                          1. 1

                                                                                                                                            ASN.1 is an interface description language not an encoding format.
                                                                                                                                            There are many different encoding rules BER, CER, DER, and PER.
                                                                                                                                            ASN.1 encoder/decoders are complex and have a long history of parsing bugs.
                                                                                                                                            I wouldn’t use ASN.1 for a new application unless it is required for interoperability.

                                                                                                                                            If ASN.1 isn’t required for interoperability then I would probably use Protocol Buffers instead.
                                                                                                                                            It’s simpler, well documented, fuzzed, and widely used.

                                                                                                                                            1. 1

                                                                                                                                              The problem with PB as far as I know is that there is nothing like CER, so there is no guarantee that 2 different implementations of PB will produce exactly the same output from the same data. What I would see instead is ASN.2 specification that would remove bloat while it would still have the nice property of having different encodings like BER, JER, XER, etc. Maybe one day I will find time to write such spec down.

                                                                                                                                          2. 2

                                                                                                                                            My vote would be for canonical S-expressions: human-readable and efficient and canonical too.

                                                                                                                                            No support for unordered sets like maps, but a) lack of ordering is part of the problem and b) they can be emulated with ordered pairs.

                                                                                                                                            1. 2

                                                                                                                                              You still need to canonicalize numbers (leading zeroes, for example) and Unicode strings.

                                                                                                                                            2. 1

                                                                                                                                              BSON! (I am 101% joking.)

                                                                                                                                              1. 1

                                                                                                                                                There’s bencode, which is almost compatible with JSON but only has a canonical form. It is also extremely simple to write encoders and decoders for; I’ve done a few in several different languages, and it’s rarely more than a day or two of work. (My most recent, bendy, took ~1 week because I wanted it to be zero-copy while still presenting a DOM-like interface) The main downside is that there is no boolean or floating point type.

                                                                                                                                              1. 2

                                                                                                                                                I still don’t understand necessity of centralized message brokers for IoT setups. Especially for home IoT (as opposed to industrial/commercial). Even if setup requires central control node, I don’t understand why this node should run message broker and processing node connected to this message broker as a client.

                                                                                                                                                If a sensor pushes data to the processing node, why it should push it via message broker and not directly? Pull is even worse: sensor should get request from message queue and then reply to the same queue, it’s worse than if a sensor would be just a listening server.

                                                                                                                                                A controllable device (i.e. light switch) should receive control message as quickly as possible, but it can stuck into thinking that it’s connected to MQTT server, but at the side of MQTT TCP connection is closed. It will not receive that message until next keepalive or something like that.

                                                                                                                                                Even putting NTP-like time protocol on top of MQTT is too quirky (and IoT devices always want to ask what’s time, they are frequently restarting, including when waking up from hibernation, and don’t have proper clocks).

                                                                                                                                                Message queues have no data aggregation built-in (unlike time-series systems, like all these monitoring systems). They do nothing interesting. Why they’re better than direct connection? I can understand message queues for “put long task for asynchronous processing”, i.e. for web apps, but what’s purpose for it in IoT?

                                                                                                                                                1. 3

                                                                                                                                                  Consider sensors that are on battery: they’ll wake up every few seconds (for things that need to respond quickly) or every minute (for simple sensors). The control node might want to send a command to it, but if the sensor is just listening, the control node would need to keep trying to connect in hopes that it manages to catch the sensor in the 50ms or so that it’s online. What happens if the sensor gets a different IP the next time it tries to connect? It’s lost.

                                                                                                                                                  How about if the sensor connects to the control node? It may be that there are multiple systems interested in readings. For example, I have a separate process that logs time series next to the control node, because home-assistant does a crap job of tagging its influxdb time series. I may also want to send commands to the sensor outside of the control node, e.g., for firmware updates. Once again, the decoupling provided by MQTT helps.

                                                                                                                                                1. 3

                                                                                                                                                  as always, feel free to submit feedback, criticism or issues!

                                                                                                                                                  1. 3

                                                                                                                                                    Just some nitpicking on dependencies:

                                                                                                                                                    • When depending on a Git repository (as you do with your colored dependency), it is a good practice to point to a particular commit or tag using the rev or tag parameter instead of the branch, as the branch’s HEAD can change but a commit or tag can only point to only one specific state of the repository.
                                                                                                                                                    • When publishing a binary (executable) crate, it is a good practice to publish along the crate the Cargo.lock. You can find the reasoning on why you should publish this file in Cargo’s FAQ

                                                                                                                                                    I will try it later though! I always complained that some prompt frameworks are using scripting languages like Python or Ruby that have slow spin-up rate, so this project seems interesting and a cool way to customize my ugly and boring prompt.

                                                                                                                                                    1. 1

                                                                                                                                                      You kind of cover this but the Cargo.lock would capture the commit that the git dependency was at when the lock file was generated. So if the Cargo.lock was checked in everyone would build against the same commit.

                                                                                                                                                    2. 2

                                                                                                                                                      I already implemented a similar tool some months ago rusty-prompt, maybe you can get some inspiration out of it.

                                                                                                                                                      1. 1

                                                                                                                                                        sure! thanks for sharing!

                                                                                                                                                      2. 1

                                                                                                                                                        My bashes (both the one that comes with Mac OS and the latest 5.0.7 from brew) seem to cache PS1 somehow, making pista break quite a lot.

                                                                                                                                                        ➜  ~ /usr/local/bin/bash
                                                                                                                                                        bash-5.0$ PS1=$(pista)
                                                                                                                                                        ~
                                                                                                                                                        $ cd git
                                                                                                                                                        ~
                                                                                                                                                        $ PS1=$(pista)
                                                                                                                                                        ~/git
                                                                                                                                                        $ cd nomad
                                                                                                                                                        ~/git
                                                                                                                                                        $ PS1=$(pista)
                                                                                                                                                        ~/g/nomad master ·
                                                                                                                                                        $
                                                                                                                                                        
                                                                                                                                                        1. 2

                                                                                                                                                          Try PS1='$(pista)'. What’s happening is that pista is getting called once, when you set PS1, and then never again. The single quotes force PS1 to literally contain the expansion, which then gets expanded (and thereby call pista) each time the prompt is printed

                                                                                                                                                          1. 2

                                                                                                                                                            Ohhh, no :( Of course. I feel like I should step far away from the computer now.

                                                                                                                                                            1. 3

                                                                                                                                                              looks like the installation instructions were faulty!

                                                                                                                                                              1. 1

                                                                                                                                                                Oh, whew, thanks for that. Now I feel slightly less stupid :)

                                                                                                                                                          2. 1

                                                                                                                                                            cant seem to replicate this, but it looks like a PROMPT_COMMAND thing.

                                                                                                                                                          3. 1

                                                                                                                                                            @hostname is nice to have if $SSH_CONNECTION is set.

                                                                                                                                                            1. 4

                                                                                                                                                              i have plans to extend pista into a library, so you could build your own prompts with pista’s functions. maybe ill add a hostname function :^)

                                                                                                                                                          1. 2

                                                                                                                                                            Can anyone comment on what open-coding is? From context, it sounds like replacing the forth definition with a reference to machine code, but I’m not 100% sure that’s right, and also can’t think through the details.

                                                                                                                                                            1. 3

                                                                                                                                                              Open coding is basically inlining with specialization. I am mostly familiar with it in the context of Smalltalk and Lisp implementations, where if the compiler can know that a particular value will be a fixnum, it can, for example, replace a call to the addition primitive (which also needs to handle bigints) with an ADD instruction. This can also avoid tagging and untagging values at every step.

                                                                                                                                                            1. 3

                                                                                                                                                              My prompt normally looks like this:

                                                                                                                                                              thequux@peewee <~> 
                                                                                                                                                              [0]$ 
                                                                                                                                                              

                                                                                                                                                              This basic prompt has stayed almost exactly the same for 15 years (there was a brief time where the second line looked like : 0 ;, but I ended up finding that intrusive) . However, it has gotten a couple of subtle features:

                                                                                                                                                              If I am in a directory with a git repo, this changes to:

                                                                                                                                                              thequux@peewee <~/path/to/git-repo> (git:master)
                                                                                                                                                              [0]$ 
                                                                                                                                                              

                                                                                                                                                              There can be other tags that appear in the prompt as well: which python venv is activated, the name of the environment I’m chrooted into (if any), what version of Ruby is active through rvm, and any shell configuration files that have changed since the last time they were loaded.

                                                                                                                                                              There are also chevrons that appear on the left side to let me know I’m in a nested shell:

                                                                                                                                                              thequux@peewee <~> 
                                                                                                                                                              [0]$ zsh
                                                                                                                                                              thequux@peewee <~> 
                                                                                                                                                              >[0]$ zsh
                                                                                                                                                              thequux@peewee <~> 
                                                                                                                                                              >>[0]$ zsh
                                                                                                                                                              thequux@peewee <~> 
                                                                                                                                                              >>>[0]$ 
                                                                                                                                                              

                                                                                                                                                              Also, if I am root, the second line in the prompt turns red. It is all designed to be unobtrusive and only show me information when it’s relevant, but also, no matter how much information is being displayed, to give me as much space as possible to type my command line.

                                                                                                                                                              If anybody is interested, the complete configuration is at https://github.com/thequux/dotfiles/blob/master/conffiles/zshrc

                                                                                                                                                              1. 3

                                                                                                                                                                I used to run the UCLA LUG’s installfest back in the late ‘00’s. Aside from my role organizing the event, I would solve all of the weird hardware that didn’t work after the bulk install, which was nearly always proprietary hardware. (To this day, I bear grudges against Broadcom and Sony for their wireless adapters and laptops, respectively). I’d explain that I’d be able to get their hardware working for now, but because of those hardware vendors’ attitudes towards Linux, it would probably break the next time they upgraded Linux, so when they replaced their laptop, they should look for one with an Intel or Atheros network card, etc.

                                                                                                                                                                I have no idea how many people continued using Linux after the installfest; most people needed it for their university coursework. However, the dozen or so people who came back with a cheap PC (out of ~100 or so people who had weird hardware that didn’t work) suggests that the strategy worked at least a little.

                                                                                                                                                                1. 1

                                                                                                                                                                  I like how alternative git interfaces are their own little cottage industry.

                                                                                                                                                                  Why have none of them been blessed by the git project itself as the new standard or recommended interface?

                                                                                                                                                                  1. 2

                                                                                                                                                                    Near as I can tell, they all involve tradeoffs. I use stgit daily, as an adjunct to branch-per-feature. I like it a lot, but would hesitate to recommend it to most developers, because when things go wonky, I usually need to fall back on fairly low-level knowledge of git. I’d imagine the same is true of the other interfaces.