1. 5
  1.  

  2. 1

    This article is 110 pages and thus I’m commenting before having read it. Forgive me!

    I’m curious about the practical implications of the hygiene problem. Certainly in an age without gensym, unwanted capture would be an ever present menace (though I have heard from programmers in lisps that lack gensym for technical reasons who aver that, even there, it’s been sufficient to choose really hard-to-collide symbols, and that they have never really suffered from unwanted capture).

    But in a modern, unhygienic language, like Common Lisp, the unanimous verdict is: ‘just use gensym.’ This is a solved problem, in other words. The learning curve is a little steeper, of course, as the macro student needs to learn about macros and also learn to recognize when they’re introducing symbols; but they do, and it’s never an issue again.

    When I was first learning lisps, this seemed awfully cavalier to me and I was very thankful for hygiene. In fact, doing most of my first macro programming in Elixir, I was thoroughly reliant on hygiene even outside of Lisp.

    However: I’ve since then devoted much of my time to macros in an unhygienic lisp and I have to say I’m seeing the CLispers’ point. Just use gensym!

    In fact, not only am I perfectly happy with gensym, but at this point in my steady progress off the path of hygiene, it’s not just the guardrails on expressive power—which is usually the complaint you hear about hygienic systems—that bothers me about Scheme’s macros. It’s actually the cognitive overhead of the name spacing and execution environments inherent to hygienic systems that I find troubling. I have much more trouble understanding what is happening to symbols as they move through the different evaluation layers! Far from feeling safer, the additional (apparent) complexity makes me feel less steady and sure of what I’m writing.

    All this to say: can it be that simple as ‘just use gensym’? The history and progress on the hygiene problem has been pursued and celebrated for decades (and it is fascinating; I will read this paper!). Have they been tilting at windmills? Should they have just used gensym twenty years ago and moved on to actual problems? Or is there more to what these people have built?

    1. 1

      The problem is it can be very unintuitive where to use gensym. And there is cognitive overhead in using it. It’s in some sense a version of the problem that C preprocessor macros have. To use them safely you just have to wrap everything in parentheses, but then you have to remember to do so in just the right way to avoid hard-to-debug surprises.

      This might not seem like a big deal but when doing something like symbolic expression substitution, I found hygenic macros really good to prevent a large class of errors.

      1. 1

        When you get to macros that generate macros it becomes very difficult to keep track of what captures what. There are also (admittedly rare) cases where a macro is intended to be unhygienic (captures a certain symbol) which doesn’t compose well with another macro which happens to use the same symbol under the hood, for example a function name it calls in the expansion that has the same name as the captured symbol.

        For example, let’s have a loop that unhygienically matches a next keyword for continuing. Then, you use another macro that expands to a call to a function next from its own namespace. This would unintentionally get captured if you just have gensym.

        Maintaining proper hygiene solves this problem.

        1. 2

          Another example:

          (let ((next ...))
            (macro-that-expands-to-loop))
          

          In this situation, the loop macro in the expansion should not capture this next, because it’s an implementation detail that that other macro uses the loop macro.

        2. 1

          I have heard from programmers in lisps that lack gensym for technical reasons who aver that, even there, it’s been sufficient to choose really hard-to-collide symbols, and that they have never really suffered from unwanted capture

          Even without gensym, you can build macros that don’t leak any symbols at all. The linked article gives an example, defining or as (effectively):

          (defmacro or (&rest args)
            (if (null args)
              nil
              `((lambda (v r)
                  (if v v (r)))
                ,(car args)
                (lambda () (or ,@(cdr args))))))