A part of me feels that this might bring something interesting to the literate programming table. The nice thing about this layout is it assumes you’re here to deeply study something. The definitions are off to the side—there for you if you need them, not taking up space in the exposition if you don’t. The discussion, especially in Talmud, tends to be much much larger than the text. That seems likely to be the case in software too, but whenever I think about doing literate programming I find a lot of dull code that really requires no elaboration.
I wind up with the same problem here I always have with literate programming, which is that most code just isn’t that interesting. What code do I want to study linearly? Usually I’m either writing new code from whole cloth or else debugging someone else’s edifice. I’m not likely to brew a pot of coffee and light some candles, sit down and really get romantic with someone else’s codebase. But maybe I should? Maybe that would make us better programmers…?
The nice thing about this layout is it assumes you’re here to deeply study something. The definitions are off to the side—there for you if you need them, not taking up space in the exposition if you don’t….I wind up with the same problem here I always have with literate programming, which is that most code just isn’t that interesting. What code do I want to study linearly? Usually I’m either writing new code from whole cloth or else debugging someone else’s edifice.
It’s not that “most code just isn’t that interesting”; it’s rather that most code isn’t that interesting to the author, and that happens because they are lugging around tons of context in their head.
For the last week or so, I’ve been on-again, off-again trying to get Factor rendering on Windows at arbitrary DPIs. Over the course of this effort, Factor’s text rendering has gone from being a black box to being boring, and I am getting there with Windows' Uniscribe API. And, honestly, both turn out to be quite straightforward; they just involve a ton of knowledge I happened to not to have originally. So when I eventually finish getting this quite right, the resulting code will be boring to me, and it’ll also be boring to people who know Factor’s text system and Uniscribe, but it’ll be completely fascinating to people who don’t know either.
One thing I think we’re not great at in programming is leveraging the fact that the screen is dynamic. I think you’re really close to hitting exactly why I think a format like this could be absolutely amazing for code. What I’d love, conceptually, is to be able to have comments at various interest levels, and to allow people of different skill levels to see different doc levels. For example, for true newbies, it’d honestly be great to show probably literally every function call’s documentation off to the side preemptively, so you can cross-reference them while you look at the code. If you know core Factor, but not the ui.text library, maybe only those functions would be shown. And if you know all of that, you probably just want any inline comments to be shown for things that are weird or unexpected (e.g., GetDpiForMonitor inexplicably returns both a DPI for the X axis and a DPI for the Y axis, even though the documentation guarantees they must be identical, which is the kind of thing I need a quick note on even if I’m an expert in a system).
That’s all doable today, but if we really thought about a Tosafot section, it could be useful in ways that would even apply for “boring” code. Part of the reason I’m able to spool up on Uniscribe so quickly is that it’s an iterative improvement on other text layout systems (QuickDraw, LaTeX, etc.), and that I am already familiar with tons of Unicode issues (normalization, CJK issues, etc.), so the docs casually gloss over tons of major design decisions that “just make sense” to me. But it’d be wonderful if I could see that information wrapping the code also. Maybe LaTeX does something neat here that I could integrate into the system, or maybe there’s this cool trick from Uniscribe that I could borrow and put into the Pango-based libraries. That might be really useful to someone who was looking to improve Factor’s text rendering systems in general, but are “boring” to me because they’re not actually relevant to what I’m doing.
And further updates on that documentation might be great, too. Maybe I note here that I’m making assumptions based on Unicode 9.0, but Unicode 10.0 changes something up (like breaking up the CJK plane because reasons). It’d be great to link to the original rationale, the new standard, and how to do the migration. (A lower-key version of this has been happening with country flags and certain emoji that display radically differently on iOS v. Windows v. Android, despite all three devices properly and reasonably honoring those code points.) Or my note on Windows' GetDpiForMonitor function might change in a world of VR where our monitors take advantage of our eyes tending to scan more side-to-side than up-and-down. I dunno. But it’d be great to be able to see how things change, and what the rationale for the changes were. If that’s why I’m there. Because otherwise, it’s boring.
So yes, all code is boring to someone. But most documentation is fascinating to the right reader, and a programming environment that seamlessly mixed the two could be phenomenal.
It’s worth noting that Talmud wasn’t written by one person. On any given page you’re seeing the original author of the biblical text, plus Rashi, plus other rabbis. What may be too much is expecting one programmer to write a literate program. It may only work if one programmer writes the center text and other programmers supply the surrounding text.
Oh, absolutely. I deleted this part of my comment, but I was going to note that you really need a networked, live, bidirectionally linked system—something kind of like Project Xanadu, but focused on code—to do what I was describing in a sane way. But maybe one day. :)
[Comment removed by author]
You’re raising a really valid point, but there are fixes, and I ironically think that they come from completely separating where comments are stored from where code is stored. In a system as dynamic as what I’m describing, doing so would let you say, “This comment was written for v9 of this code, but you’re looking at v15; here’s what the code looked like at v9. Feel free to update this comment or mark it as inaccurate.” We’re honestly heading that direction already with commit messages and hg annotate/git blame; it’d make sense to just own that more completely.
I’d love to hear about your experiences with TeX. I took a couple months last year to work with plain TeX and found it quite enjoyable.
I came into this in the early 2000s, so my first exposure was to LaTeX, and I didn’t think it looked all that nice. Later I came upon ConTeXt, which I found much easier to make look the way I wanted. It’s sort of assumed with ConTeXt that if you need to do something really fancy, you’ll head to either plain TeX or Metafun (or TiKZ). So I had the impression that ConTeXt was a lot closer to plain TeX than LaTeX. And I think that planted the seed for me to learn plain TeX. In between, I picked up troff (Heirloom Troff is quite nice actually) and looked at a few other things (Docbook, Lout). I haven’t had much occasion to do much typesetting lately, but if I were going to, I’d probably be choosing plain TeX absent other information.
Another reply mentions displaying age markers with comments; these look like obsolete-page markers in NethackWiki (which sometimes has source code discussion, too…) Detailed game wikis also seem to mark clearly whether a paragraph is about game mechanics, game strategy, in-universe background, outside cultural references, or technical limitations of implementation.
Maybe in some kinds of code it is a good idea to have a convention of separating generic domain knowledge, local organizational practices, technical approaches across the codebase and local technical decisions; sometimes even the rate of change can be predicted.
I guess a separate difference of program code as opposed to biblical text is «value density» — those who study Talmud care about details of each sentence; but there is a lot of code where the details are just not worth caring about; and we value not needing to read code in order and not needing to pay attention to everything to find the things we are looking for.
You’re probably talking about Docco and its spinoffs, Pycco (I’m the maintainer); Marginalia and others.
Literate programming is one of the biggest failed memes in the last half century of software engineering.
Consider: if the point of programming is to leverage abstraction, what is the bloody point of knowing the details about how something was done? We all by construction do not care.
My issue with literate programming is that it makes refactoring much harder and the weaving of LaTeX (ugh) and whatever language can get very annoying to deal with.
Well, it depends what you consider Literate Programming. If we consider it as defined by Knuth you’re right on the complexity, as the original tool functionality (CWEB and noweb) never was incorporated to IDEs.
But you can use markdown to do Literate Programming and it works nicely (see codedown). Obviously, is not straightforward to move a 1 million lines of code to this, but starting a new program with markdown as literate programming tool is easy and a good way of starting with literate programming.
This is mostly regarding the original Spolsky post, but if any piece of writing requires explication to its intended audience, the author has failed his task.
Considering programming, I expect the audience to evolve. What’s informative for the next maintainer when they’re coming up to speed is likely to get in their way as they maintain the code longer.
Suppose that I accept that:
if any piece of writing requires explication to its intended audience, the author has failed his task.
but this doesn’t help us to solve the actual problem.
Today almost all the code written requires additional comments or documents to reach its intended audience, so… what should we do to solve the current mess that is reading other people code?
I recently was doing a deep dive into a specific topic in approx. algorithms and spent countless hours trying to find what the difference was between literally five almost identical presentations of the same algorithm. If the authors had spent a couple hours of their time making their papers more carefully reflect past literature instead of just spamming footnotes, I am sure the hours saved by the many others reading their papers would have been at least ten times as much–meaning moee time for people just like them (not even talking about non-researchers like me) to work on other problems instead of just getting up to speed.