1. 60

  2. 5

    I agree.

    The right insight can drastically simplify code. I’ve heard all of the following distinct cases described as “clever”:

    1. Excessive use of obscure language features (bad, as covered in the article).
    2. Excessive use of indirection (also bad).
    3. Relying on a mathematical or conceptual insight that, while not obvious, is easily understood once pointed out and genuinely simplifies the code (a pure good).
    4. Relying on a mathematical insight that is not easily understood but that simplifies or speeds up the code (a tradeoff – comments can help here).
    5. Relying on debatably obscure language features. The kind of code that might not be immediately understood by someone new to the language, but that would be standard to experienced devs. (again, a tradeoff, a judgement call, and an invitation to the question “what feature set should we assume knowledge of?”)
    1. 11

      I run across #5 quite a bit as someone who teaches computer science. My undergraduate students almost invariably consider code that uses Python list comprehensions to be unnecessarily clever, or even code-golfing. Why not just write an explicit loop that shows what you’re doing? But among more experienced Python devs, they’re very idiomatic.

      1. 6

        Relying on debatably obscure language features.

        This is the trickiest and most common, IME. No language feature is truly “obscure”, and if the reader doesn’t know about it it’s a deficiency in their knowledge of the language. Which features are “obscure” is in the eye of the reader, and so it’s a team-wide effort to decide where you land.

        1. 3

          so it’s a team-wide effort to decide where you land.

          This is key. Most modern languages have many features and support multiple paradigms, so that in practice, in a team setting, what you are really programming in is a subset of the language. I think good teams document that subset, and have a shared understanding of why they use it, and why they avoid the parts they do. This includes linter rules and style guides, as well as higher level rules. In unhealthy teams that subset is implicit and followed inconsistently. Of course you can say everyone should be an expert (and that is a good goal), but in practice, with new hires, people ramping up, people contributing from other teams, etc, it’s not realistic.

        2. 4

          Regarding point 5, I prefer language features a newcomer might not know, but can read enough to understand.

          In clojure a newcomer might not know zero? is a function, but when they see it, it becomes obvious. Similar enough with threading, people see the value in it when they see code with it.

        3. 4

          Dr. Talia Ringer’s work on proof repair is an interesting potential path to addressing the problem of fragility. The notion of proof repair is to enable automated “repair” of proofs when a program changes. Dr. Ringer’s thesis (link to preprint) is on a practical proof repair system for Coq. The “practical” part has been a problem in this area for a while.

          In general, work like this which makes interactive theorem proving more accessible could help in the future to identify confidently exactly when a change has broken the assumptions of what Hillel calls “insightful code” here.

          1. 6

            I assume you’re doing the same thing I’m doing where we’re referring to her as Dr as much as possible to really maximally celebrate that she got her PhD

            I also wish I knew enough Coq to understand what’s going on here

            1. 1

              That is absolutely what I’m doing, yes!

          2. 3

            The distinction he’s trying to draw is a bit too fuzzy to draw directly, and is at best a spectrum, because every piece of code is an interface between the language it’s written in and the problem domain. Nearly all of his “insightful” examples are actually cases where specific language features (often not understood by outsiders) happen to match relatively well-understood problem-domain features. This is probably at least partially a side effect of writing for a general audience of programmers (who are more likely to know enough python or haskell to get by than to understand specific quirks of an obscure domain), but it’s also a very real problem for maintainability concerns, since maintenance in big companies is often put on the shoulders of junior engineers who aren’t experts in either the domain or the tooling. It muddies the water because duff’s device, in its original incarnation, was exactly this kind of problem: written in an environment where everybody knew C quite well, it was a necessary optimization for a particular piece of code that ran too slow otherwise, & while everyone recognized it was clever at the time, it only gained its low reputation when C optimizers got good enough (and clock speeds got high enough) that optimization tricks no longer needed to be in the toolbox of your average developer.

            In other words, rather than two classifications of ‘cleverness’ there are two systems you can be clever in interacting with – the problem domain and the language domain – and to the degree that one is clever toward one of these domains, the resulting code is fragile with respect to changes in that domain and obscure with respect to a lack of knowledge about that domain.

            To illustrate, take an example from python. Expecting an expression with division in it to be an integer division expression if all of its components are integers used to be as normal in python as using list(set(x)) to remove duplicates from some list x – in other words, not clever at all, and not even particularly python-specific (since many languages, including java and c, work this way). With python 3.x, dedicated integer division operations were introduced and an underlying distinction within numeric types between integers and non-integers was eliminated – so the behavior of division with respect to integers changed drastically, in a way that breaks a lot of third party code. So expecting division to work the same way in python as it does in c was, essentially, “too clever” with respect to the massive changes to the language being undertaken, simply because those changes happened to make this behavior fragile.

            This is almost exactly the same as a problem being redefined & requiring a code rewrite. The difference is that, in today’s professional software environment, there’s an expectation that third party interfaces (including but not limited to language features) will change slowly or not at all while the problem definition you’re working from will probably change multiple times per sprint, being contingent & based on faulty data in the first place.

            Python changing the definition of division is not structurally different from your PM requiring that a parameter should be able to come as a list or an object; you just don’t expect the former as often, so there’s an expectation that developers can and will over time learn all of the quirks of their language.

            This part of maintainability has a lot of intersection with other parts (like compatibility). Like, the difference between “language feature” and “implementation quirk” comes down to whether or not it’s standardized (and whether or not ostensibly standard-compliant implementations obey that standard). The more independent implementations there are, the longer it takes for core features to mutate, but at the same time, if you have a single dominant implementation, you can expect obscure quirks to be more widely applicable. Sometimes, the value of using an implementation quirk is enormous – for instance, GNU Awk has a non-standard feature for streaming to an open pipe, and so if you decide that you’re only ever going to use GNU’s version of Awk, you can build things like spawning HTTP requests in the background into your data-processing awk script, preventing you from needing to involve a heavier language or framework.