1. 9
  1.  

  2. 12

    Chen’s blog post is interesting in both what it references, McIlroy critiquing Knuth, and in what it misses in that exchange.

    In short, in 1986 Jon Bentley asked Donald Knuth to demonstrate literate programming by implementing a word-count program which would then be critiqued by Doug McIlroy. Knuth delivered a beautiful example of literate programming in Pascal. 10 pages worth. McIlroy, in addition to his critique, delivered a six-segment shell script that accomplished the same thing without intermediate values…a purely functional implementation as Chen describes it.

    McIlroy, among other comments, ends his critique with:

    Knuth has shown us here how to program intelligibly, but not wisely. I buy the discipline. I do not buy the result. He has fashioned a sort of industrial-strength Fabergé egg—intricate, wonderfully worked, refined beyond all ordinary desires, a museum piece from the start.

    That’s the background.

    Chen takes up the topic because he’s intrigued by McIlroy’s solution because it’s purely functional, wondering how he’d do the same today. He writes his solution in Haskell in two variations: “standard” and literate. As a Haskell implementation, it’s effective. Chen then discuss the advantages of both and falls on the side of “standard” rather than literate. Had he left it at that, it would be an interesting bit of Haskell.

    A curious exchange in the comment section brings the discussion back to McIlroy’s critique of Knuth. Dorin B takes Chen to task for misunderstanding McIlroy’s point:

    You missed the point in McIlroy’s sollution: to use reusable components.

    Chen then replies:

    No, I think I illustrated exactly the point that McIlroy was making, and I believe that if you emailed him, he would completely agree with me today. … Note how every single line in my Haskell program is in fact a reusable component.

    Chen completely misses Dorin’s that for McIlroy’s reusable components isn’t about functions or sub-routines but composable tools. Dorin’s right.

    In the interview Chen posted with McIlroy the question that segues into a discussion of his critique of Knuth’s solution begins with a discussion of how pipes effectively invented the concept of the tool. McIlroy says:

    McIlroy: Yes. The philosophy that everybody started putting forth: “This is the Unix philosophy. Write programs that do one thing and do it well. Write programs to work together. Write programs that handle text streams because that is a universal interface.” All of those ideas, which add up to the tool approach, might have been there in some unformed way prior to pipes. But, they really came in afterwards.

    MSM: Was this sort of your agenda? Specifically, what does it have to do with mass produced software?

    McIlroy: Not much. It’s a completely different level than I had in mind. It would nice if I could say it was. (Laughter) It’s a realization. The answer is no. I had mind that one was going to build relatively small components, good sub-routine libraries, but more tailorable than those that we knew from the past, that could be combined into programs. What has… the tool thing has turned out to be actually successful. People just think that way now. That’s providing programs that work together. And, you can say, if you if stand back, it’s the same idea. But, it’s at a very different level, a higher level than I had in mind. Here, these programs worked together and they could work together at a distance. One of you can write a file, and tomorrow the other one can read the file. That wasn’t what I had in mind with components. I had in mind that … you know, the car would not be very much use if its wheels were in another county. They were going to be an integral part of the car. Tools take the car and split it apart, and let the wheels do their thing and then let the engine do its thing and they don’t have to do them together. But, they can do them together if you wish.

    MSM: Yeah. I take your point. If I understand it correctly, and think about it, a macro based on a pipeline is an interesting thing to have in your toolbox. But, if you were going write a program to do it, you wouldn’t just take the macro, you’d have to go and actually write a different program. It wouldn’t be put together out of components, in that sense.

    McIlroy: So, when I wrote this critique a year or two ago of Knuth’s web demonstration. Jon Bentley got Knuth to demonstrate his web programming system, which is a beautiful idea. …

    Now, in 1968, I would have thought he was doing just right. He was taking this sub-routine and that sub-routine, and putting them together in one program. Now, I don’t think that is just right. I think that the right way to do that job is as we do it in Unix, in several programs, in several stages, keeping their identity separate, except in cases where efficiency is of extreme importance. You never put the parts into more intimate contact. It’s silly. Because, once you’ve got them there, it’s hard to get them apart. You want to change from English to Norwegian, you have to go way to the heart of Knuth’s program. You really ought to be able to just change the pre-processors that recognize this is a different alphabet.

    For Chen to then argue that his Haskell implementation illustrates exactly McIlroy’s point shows Chen either didn’t read what McIlroy had to say about it, or doesn’t understand it. That’s not to say McIlroy is against functions, sub-routines or software toolboxes. But that’s not the point McIlroy was making.

    Of course Chen isn’t alone in misunderstanding Unix and what Thompson, Ritchie, Kernighan, McIlroy and many others achieved with it. In a nutshell this is what distinguishes many BSD users from Linux users.[1] BSD isn’t merely about POSIX, nor is it about avoiding Windows and other proprietary software (important as those goals may be). BSD is mostly about the Unix philosophy.[2]

    On the whole, Chen’s discussion of literate vs “standard” program is interesting. As a Haskell programmer, I find his solution informative. As commentator on McIlroy or the Unix philosophy, I’ll look elsewhere.

    [1] That’s not to say many Linux users aren’t interested in the Unix philosophy or that all BSD users are Unixphiles. Setting aside criticisms about security, implementation and what have you, the issue many Linux users have with /systemd/ is isn’t Unix-like.

    [2] Yes, the various BSDs differ a bit on how that looks and how rigorously to pursue it.

    Edit: Fix formatting.

    1. 2

      I think that the right way to do that job is as we do it in Unix, in several programs, in several stages, keeping their identity separate, except in cases where efficiency is of extreme importance.

      We often must ditch this lots-of-separate-programs approach whenever efficiency is of more than negligible importance.

      1. 0

        I admit I know nothing of the wider context; all I know is what you’ve posted here. But from what you’ve written it sounds like Chen is presenting the 21st century McIlroyian view which may not be the same as the original in concrete terms but is the same spirit rebased on today’s technology.

        1. 3

          I think Chen is trying to cast McIlroy that way, but for it to be the 21st-century version of McIlroy’s argument, reusable software components would have to come after tools. But that’s not how it went. Indeed, McIlroy says as much in his critique of Knuth:

          Now, in 1968, I would have thought he was doing just right. He was taking this sub-routine and that sub-routine, and putting them together in one program. Now, I don’t think that is just right. I think that the right way to do that job is as we do it in Unix, in several programs, in several stages, keeping their identity separate, except in cases where efficiency is of extreme importance.

          Chen is saying more than is warranted based on the support he provides (the linked interview). It’s one thing to write something like: “In the same spirit as McIlroy’s reusable tool approach…” It’s another to write:

          No, I think I illustrated exactly the point that McIlroy was making, and I believe that if you emailed him, he would completely agree with me today.

          Again, that’s not to say McIlroy would disagree with a reusable-component approach to software development.

          The point is McIlroy was making a very specific critique of Knuth’s program based on value of tool over software reuse and composition and Chen completely missed it. And then asserts McIlroy would agree with him that writing software with reusable software components is point McIlroy was making.

          Chen misunderstands McIlroy, and worse, imputes his misunderstanding to McIlroy.

      2. 1

        This sounds like a slightly more complicated FizzBuzz to me.

        1. 0

          The thing that struck me was the simplistic view of a “word” both Knuth and McIlroy took. As Knuth said:

          Let us agree that a word is a sequence of one or more contiguous letters: "Bently" is a word, but "ain't" isn’t. The sequence of letters should be maximal, in the sense that it cannot be lengthened without including a nonletter.

          Remove that simplification, and Knuth, who is already writing code, can just modify the code that determines what a “word” is; McIlroy would have to break down and write a program. When I did the “word count program” for my 2014 NaNoGenMo it took some work for me to determine what a “word” is. It would be interesting to see what both would have written in such a situation (where “isn’t” is a word).

          1. 6

            Nah, you’d only need to replace (from eyeballing it) the first part of the pipeline:

            > tr -cs A-Za-z '\n'
            

            If you wanted to support more clever words (or punctuation-heavy languages like Klingon), you’d add punctuation to the set there, using some of the bits from the man page. If you wanted to prevent words that included numbers or punctuation, perhaps a regex and sed would do it.