I think rel="me" is a neat little concept. Had no idea that anyone was misunderstanding it as some sort of author attribution.

I’m glad the author makes the point that the purpose of markup isn’t to appease Google. Still, I find it strange that so much supposedly “indie” web stuff seems to revolve around silos, and Twitter in particular. I’ve implemented some of it on my own site, but now that Mozilla Persona is dead their auth ideas all seem to rely on “log in via twitter” (or some other proprietary service) :(

But the firm’s latest rankings, released last week, show Swift dropping to 11th place. Kotlin slipped from 27th place to 28th place. Though notable, slipping one place in one quarter doesn’t mean that Swift and Kotlin are in decline or even that they’ve peaked. “In general, we caution readers not to assign too much weight to small changes in the rankings; the differences between one spot or another, in general, tend to be slight,” RedMonk co-founder Stephen O’Grady wrote in a blog post analyzing the findings.

They literally quote someone who says that articles like this don’t mean anything…

I’ve always stuck with cron for scheduling. On my own laptop I use Task Spooler to queue up expensive commands so that they run one at a time in the background (e.g. large downloads and network transfers; as well as commands which may interfere if run concurrently)

Never heard of it, but it seems like a super interesting approach to interactive environment. I cannot help but remember this Bret Victor’s talk about how we have been programming in almost-anachronistic ways with no innovation in the interfaces.

I think there are actually two distinct forms of “reusability” in the wild, which I like to distinguish between.

Some things are reusable because they’re so small and simple that they can slot into solutions for all sorts of problems. Examples include unix utilities, JSON/protobuf/etc., arrays, linked lists, filesystems, SQlite (RDBMS servers like Postgres are debatable, I personally wouldn’t call them “simple”), key/value stores, etc.

This is a good form of reusability, since this sort of code tends to be solving a well-specified (although abstract) problem, and can often be considered “finished”. Pulling out such functionality from a larger program hence reduces the amount of stuff we need to care about.

Other things are reusable because they have so many hooks/options that we can configure them to plug into all sorts of solutions. Examples include stereotypical “enterprise ready” software, towering monoliths of code like GCC and GHC, “frameworks” and “ecosystems” like Symfony, Drupal, Wordpress, etc.

This second form of “reusability” is often underisable. We need to go out of our way to support other use-cases, which may not even exist. This sort of software is often modelling some concrete, domain-specific concern like “users” or “reports”, which are informal enough that the decisions encoded in this implementation are probably unsuitable for other applications, and hence the need to keep adding more and more parameters and overrides.

I think there’s a correlation with how functional programming and object oriented programming tend to be practiced too. FP libraries usually provide some abstract entity, which may be simple, self-contained and reusable in the first sense. Whilst OOP libraries often provide interfaces to some concrete entity, like a Web service, with hooks and parameters to make it reusable in the second sense.

I think this may be a consequence of their approaches to code layout, the expression problem, etc. OOP limits what we can do (the set of methods), but subclassing makes it easy to override and alter how those things are done. FP limits what there is (the data constructors), but makes it easy use those things in new ways.

Extensibility and re-usability are potential goals for system boundaries only. ..creating something that’s re-usable is pretty inherently about creating something that’s a system boundary, to some degree or another. And if you’re knowingly working on a system boundary… you’re knowingly working on something that supposed to be re-usable already. It’s pretty redundant to hail it as a design goal.

This is great. I’ve been collecting anti-reuse, anti-abstraction links. Everytime I add one to my collection I’m going to share the whole thing.

• “Prefer duplication over the wrong abstraction.” – Sandy Metz

• “Write code that is easy to delete, not easy to extend.” – https://programmingisterrible.com/post/139222674273/write-code-that-is-easy-to-delete-not-easy-to

• “Abstraction, in and of itself, is not harmful. On the contrary, it’s necessary for progress. What’s harmful is relying on impenetrable barriers to protect our precious programmers from hard problems.” – Tyler Treat

• “If the abstraction you wrote requires someone new on the project to spend hours understanding it, then you’re probably doing something wrong.” – Vlad Zelinschi

• Disadvantages of code reuse

Sounds like a pleasant experience. One thing I wanted to point out is that many FP practices/principles (single assignment, returning fresh data instead of mutating, etc.) can be followed in almost any language, in case any JS or TypeScript users didn’t want to switch to something like PureScript. I find that they tend to be good ideas even on their own, as long as it’s not going against the grain of surrounding code.

I agree with the overall message, but I find some of the particulars to be non-issues. Personally, I instinctively stick to the following, and it avoids a lot of issues without having to switch to a “big boy” language quite as soon/often:

• Always quote variable substitutions, e.g. "$foo" never $foo. I find it disingenuous when someone (not this author, but it’s common enough) complains about e.g. filenames with spaces, says they don’t want to quote everything because it’s ugly/hassle/etc., then claims to prefer a language like Python/Go/etc. where all strings must be quoted. Why is it ugly/hassle in a shell script but not in those other languages? Don’t think about it as “adding quotes”, just default to treating shells like any other language.
• Never use word splitting. If the shell has arrays then it’s basically an anti-feature, so I just pretend that it doesn’t exist so that I’m never even tempted to use it. Again, just quote everything ;)
• Treat each shell as a different language. The difficulty of making scripts “portable” is a common complaint about shells, which is valid but again rather disingenuous if the “solution” is to enforce a particular dependency like Go (for compiling, at least) or Python (which is usually shorthand for something like “CPython > 2.7 && < 3” or “CPython 3.5+ with these packages…”). If you’re going to force a particular interpreter for Python/etc., then why not enforce a particular interpreter for your shell scripts? If it’s fine to write a Python script, then it’s fine to write a “bash 4+ script”. Who cares if it doesn’t run on zsh or dash? Python scripts don’t run on zsh or dash either.
• These days I tend to go even further and use Nix to bake-in the versions of bash, grep, sed, etc. that I’m using, with a pinned nixpkgs version. I also do this for python scripts, etc. as well. If I have a collection of scripts, I’ll also have Nix run Shellcheck on all of them as part of their “build”; that can help catch things like undefined variables too. I don’t tweak the settings: just abort on everything, and add # shellcheck disable=SC123 comments as appropriate.

Don’t use N computers when 1 will do.

This is so important, because people seem to forget that computers are fast. Really fast. Absolutely, mind-mindbogglingly fast. CPUs operate on a level of granularity of billionths of a second, while humans can barely sense thousandths of a second.

The only reason computers are “slow” is because most software is crap. I mostly blame Windows and the Web, since they’ve mislead average computer users into believing they have to wait for bloat to run (rather than regularly shouting at developers to profile their code).

An average laptop running F-Stack and a decent web server could host a website and handle millions of concurrent requests per second. Distributed systems on AWS are nice and all, but they’re way harder to build than systems that run on a single server. And they’re far harder to profile and optimize.

/rant

Out of curiosity, I wonder how the author managed to handle concurrency in SQLite, since it locks the whole DB on writes. He mentions writing multiple SQLite databases, but I’d be interested in more details.

Linters are great for running as git pre-commit hooks. I like the “out of the box” ideas given in this article: i.e. it’s not just about code style.

It’s important to keep these fast though. I treat slow linters like integration tests: have them run on a build server, and only run them locally when reproducing/debugging a failure.

My own collection of PDFs is getting a little ridiculous:

$ find Documents/ -iname '*.pdf' | wc -l
2385

Most of these are stored with the original filename, which can be nice in some ways (e.g. I’ve often tried to save a paper off ArXiv only to be told the file already exists!), but as you say they’re often useless from a human/semantic perspective.

I’ll certainly take a look at PaperBoy to help with this.

For anyone interested, I wrote up some tips for managing PDFs, including links to a bunch of scripts (and a few snippets of my own): http://chriswarbo.net/projects/pdf-tools.html

It was written a few years ago, so unfortunately some of the links are broken (e.g. the Nix packages have probably moved to http://chriswarbo.net/git/warbo-packages.git now)

I make heavy use of assertions in many languages. A really easy way to get more context is to just dump out a map/dictionary of values. For example, in Python:

assert foo == bar, repr({
  "error": "Foo and bar should not be equal",
  "foo": foo,
  "bar": bar,
})

It’s easy enough to wrap this into a function (although that may lose line number information, depending on the language), e.g.:

def check(bool, info):
  assert bool, repr(info)
  return bool

I do a similar thing in other languages by dumping out the context information as JSON. One thing to be careful about is whether that info itself will cause problems: huge values are annoying, but infinite/cyclic values may break the error message itself! Also in a lazy language we may have values in scope which haven’t yet been forced, and trying to write them out as JSON may trigger an error. I find it best to code defensively in these situations (assertions are useful precisely when we are wrong about what’s happening!), e.g. not including values which are only accessed after the assertion.

For those trying to learn Nix, I highly recommend learning the Nix language using the REPL. If you’re on a 1.x version of Nix (or NixOS < 18.03) you can use the nix-repl command provided by the nix-repl package, e.g.

$ nix-shell -p nix-repl --run nix-repl

If you’re on Nix 2.x or above (or NixOS 18.03 or later) this is now built-in to Nix, just run:

$ nix repl

For Nix 2.x there’s also the --show-trace option which will give you more debug info if there’s a failure.

One gotcha with the REPL is that it will sit waiting for input if it’s given an incomplete expression, even if that expression is from an external file! For example:

$ echo "''unterminated string" > foo.nix
$ nix repl --show-trace
Welcome to Nix version 2.0.4. Type :? for help.

nix-repl> import ./foo.nix

This will hang, presumably waiting for the ''. This is usually obvious for things like strings, but less obvious for e.g. expressions which contain ; like with foo; bar or assert foo; bar (those semicolons aren’t terminators, they’re separators, hence the following expression (bar) is required, or else it’ll hang like above)

Oh boy. Here, have a hopefully helpful anecdote…

I spent a lot of time, sweat, tears, and pain deciding to “properly” learn Nix to set up armokweb, aka lobste.rs plays Dwarf Fortress. I’ve yet to do a full writeup/postmortem (although I intend to), but I finished the first iteration of that project with a mostly-positive view of NixOS from no experience whatsoever.

First, let me get it out of the way: the documentation often sucks and you have to read other packages that do a similar thing to the package you want to write to create custom packages at all. There’s not really a “Nix package cookbook” like there should be. Nix Pills gets you part of the way there, but my perception is that the whole nixpkgs contribution process operates on screwing up and learning from maintainers who have more experience than you.

nixpkgs is beholden to how well the community maintains it, but this is similar to Arch’s situation with the AUR. If there were fewer maintainers, the AUR would not be as useful. Some Nix packages (including the dwarf-fortress one, which I ended up contributing to) had breaking bugs. I’m not sure I’m qualified to argue for how “mature” nixpkgs already is, but it’s definitely getting better.

On a more positive note, I like Nix’s immutable packages and declarative approach to configuration. I tested armokweb in a Nix VM, and it was near-trivial to refactor it so it ran in a container that could be included in a Nix config with two lines of code. Nix’s strong point, IMO, is that it encourages you to produce artifacts (Nix code) that can be used to reproduce a system configuration on another machine, or in a container, or in some other context within the system. Coming from Debian, CentOS, and Arch, it’s a godsend that this is included in the OS and you don’t have to separate the steps of configuration and reproducing that configuration for others to use as much as you would with something like Ansible. It’s also great that you can mix rolling and non-rolling release models all you want. Your entire system doesn’t have to be “unstable” - just the packages and maybe their dependencies that need to will be if you decide to install an “unstable” package. There are no dependency diamonds that cause apt to complain that the wrong version of libc is installed if you accidentally add the wrong mirror.

Nix the language is sort of like a weird Python/ocaml hybrid, but the lazy evaluation approach works really well for package management. Derivations stringify to their Nix store path, which is the content address of that package (and, when they’re depended on, they get loaded, either from local build via cryptographic content-hash or from cache). It’s why the NixOS cache can exist as effectively just that, a cache that just makes installing packages go faster because they’re on a nearby server instead of needing to be rebuilt locally.

Overall, I’m a fan of NixOS. The entirety of system configuration is filling in the blanks in a playbook that you can share with others. Packages have strong guarantees about their integrity. Nix the language is a little weird but workable once you’ve seen enough of it. Containers just use lxc and lots of hardlinks to function. I’d like to see deeper integration with ZFS for managing the Nix store. Currently, garbage collection is done at a user-facing FS level, when Sun Microsystems solved this problem years ago at the block level.

I think there’s plenty to like about what NixOS provides, with the caveat that they can do all this better down the road. I feel like it would be really informative for them to refactor the Nix store so it’s able to take advantage of the features of ZFS. OpenIndiana already uses ZFS to manage system “generations” in a similar way to how NixOS uses symlinks and hardlinks. They would be able to make their OS better if they learned more from IllumOS, but I have no doubt that they could tackle that if they wanted to.

Looks very nice. I’m currently using Hydra but it’s annoyingly heavyweight (postgres DB server, user account/authentication system, configured via destructive Web UI, no CLI, etc.). I think I’ll be switching to Laminar at some point ;)

I really like ATS as a proof-of-concept, and as a way to avoid falsehoods like ‘low level == unsafe’, ‘safe != fast’, ‘C/C++ is the only way to do X’, etc. It’s a bit too verbose and fiddly for me (I’ve been spoiled by high level, garbage collected languages like Agda, Idris, etc.) but it’s nice to see people (other than its author) playing with it, since that’s the only way to find and smooth-over the rough edges (like with the atspkg tool described in this article!)

I really want to love NixOS: the ideas, the tools, how things are supposed to work… All they propose sound like future to me. Be able to have my config, which defines how I want my computer to behave, and just plug it in all the machines I may need to use sounds mindblowing.

And personally, I am finding the learning curve to be steep as hell. Not only because the documentation seems to assume that the one reading is slightly familiar with the environment and how things work, also because I need to modify certain habits to make them work with NixOS. For example, one of the must-haves for me is my Emacs configured as I like. I can tell Nix to clone my Emacs configuration to the home folder, and it should already be able to start downloading the packages it needs; but in reality that is not trivial because it seems to expect the packages to be downloaded from the Nix configuration instead of the Emacs one (to ensure the system to be deterministic, it makes absolute sense). I am used to have everything available from everywhere, but NixOS has most things isolated by default to keep the purity.

I will keep on fighting with stuff until I find things out, but I am sure that as the project grows all these corners will be polished to make it more accesible to newcomers.

Git via email sounds like hell to me. I’ve tried to find some articles that evangelize the practice of doing software development tasks through email, but to no avail. What is the allure of approaches like this? What does it add to just using git by itself?

Haskell’s QuickCheck is my test framework of choice. There’s a really nice library called LazySmallCheck2012 which I’ve used a few times, but it seems mostly unmaintained (I patched it to work with GHC 7.10, but the pull request has sat unmerged for years). Maybe some idea in that direction would be useful?

Let’s say we need to generate some data of type (Bool, Either String Colour). With QuickCheck and SmallCheck we would choose a value (at random or systematically, respectively), like (True, Right Blue), and pass this to a test.

The idea of LSC and LSC2012 is that we only make choices when we have to. We do this by effectively running our test on the value undefined. If the test passes, we know it will work for all possible inputs. If it fails, we’ve found a (minimal) counterexample. If it throws an exception, it must have tried branching on the value we gave it; in which case we re-run the test on all those values which make one more choice than last time, e.g. (undefined, undefined). If we get another exception, say from the second element, we would re-run with both (undefined, Left undefined) and (undefined, Right undefined). We keep doing this, adding specificity if there’s an exception, until either the test passes, we find a counterexample, or we reach some depth limit.

The effect seems to be rather like using logic programming.

What would you like to have improved in generative testing in general?

• Smart ways to generate recursive data structures with low risk of them blowing up exponentially as the size parameter increases

• Smart ways to direct generator distributions to problematic inputs.