1. 38

  2. 47

    TeX is a really strange program by modern standards. One of my favorite tidbits about it is that you can launch external programs and read their output, but that facility works by writing the command line to a magic-number file descriptor and then reading from that file descriptor.

    I have been whatever the opposite of a fan is of LaTeX for a long time. I learned ConTeXt because I got sick of the magic libraries, noisy syntax and general ugliness of LaTeX. I think a lot of computer scientists and programmers hold LaTeX’s output in higher regard than it deserves, because Knuth is a genius and studied typography and Lamport’s a genius and invented LaTeX or whatever. But the defaults for LaTeX are hideous to anyone with eyes and Knuth’s fonts are not amazing outside math.

    ConTeXt absolutely makes more sense than LaTeX. If you need a powerful facility for generating quality PDFs for print, like the idea of using TeX and the cost, definitely check it out. A downside of ConTeXt is that it sort of assumes that you really know what’s going on with TeX for deeper customizations. This sent me down a rabbit hole with TeX.

    The basic ideas of TeX are definitely interesting and powerful. The problems (in my opinion) are twofold: first, the primitives you have access to just don’t seem to be powerful enough for a lot of modern problems, and second, you’re a bit hamstrung by the design constraints of the system itself. Regarding the former, (for instance) it’s rather difficult to set things up so that TeX figures out the sizes things should be, but also is able to retain those sizes across page boundaries. TeX wants to create boxes and fill them in, and it’s got a separate asynchronous thing that ships out pages when it has enough boxes to fill them. Communicating between those facilities is tricky. This makes things like facing-translation hard. At the same time, the macro facility in plain TeX, unconstrained by LaTeX’s conventions, is much better for human readability.

    For efficiency reasons in the early modern era, TeX wants to process the document only in a forward direction. This leads to the second problem. There’s no way (in one pass) to communicate backwards to earlier pages and there is no data structure in memory representing the document as a whole. This is why it takes many passes to generate a LaTeX document: each reference’s location gets recorded in the first pass, and on the next pass LaTeX can read the file with the references to generate the appropriate text. Of course, when a reference changes from “(?)” to “(Subrahiman, Mellish, Gazdar et. al 1977)”, that can change how many boxes fit on the page, so that can necessitate another pass. Having to do work in multiple passes is so common in TeX that there’s a primitive operation for creating a side file and writing commands into it.

    The mouth/gullet/stomach thing has to do with macro expansion and what knowledge TeX will let you have and when. The process overall has an intriguingly asynchronous flavor, that it’s in horizontal mode building this line, then enters vertical mode building these paragraphs, and eventually has to engage the page shipper because you have too many paragraphs. But while I’m sure that it was easy for Knuth to keep all this straight in his head, it’s quite unstraightforward for mortals. And of course it makes machine-processing of TeX source by anything other than TeX nearly impossible.

    Part of the popularity of LuaTeX is that it makes it really easy to do things that, while possible with plain TeX, would be incredibly onerous because it just isn’t a modern programming language. I had a macro for LuaTeX that would take some SQL, run it against a database, and then emit syntax highlighted SQL and a nicely formatted table with the real result of running that SQL. This entailed loading a common Lua library and a common TeX library for formatting code, and just sort of gluing it all together with Lua, it was simple. Conversely, the most complex thing I’ve ever tried to do with plain TeX is probably to make a résumé with colored rules and real fonts. Having a real modern language with real data structures at your disposal helps immensely.

    Finl is not the first bold TeX replacement I’ve heard of; lout comes to mind, and got pretty far along before it got abandoned. There are so many hard parts to this problem: dealing with PDF, fonts, page layout in full generality, math. It seems like usually one of them is fatal to a new approach. I’m often tempted to give up on this as a concept and just buy Prince XML or just say everything’s going to be HTML from now on. But sometimes you just want to see a good-looking drop cap, or you need to give someone a PDF or something. Anyway, I wish the author well on this extremely long and complex journey!

    1. 28

      Of course, when a reference changes from “(?)” to “(Subrahiman, Mellish, Gazdar et. al 1977)”, that can change how many boxes fit on the page, so that can necessitate another pass.

      Many years ago, I found (and have since lost) a .tex file someone designed to never converge, because the references would dance between page boundaries.

      1. 28

        It is well you lost this cursed object.

        1. 8

          Good news! I found it again: https://tex.stackexchange.com/a/79699

        2. 9

          Not quite the same, but I’ve had something similar happen on the web sometimes. Hovering over an element will cause it to move out from under the mouse, following which it will move back, so it would jump back and forth and flicker indefinitely.

          1. 0

            I find that so aggravating!

        3. 9

          Reading the TeX source code is a treat: http://brokestream.com/tex.pdf

          It is probably the only piece of software that’s not trivial yet is laid out so logically that I can figure out how something is done in:

          1). A language I don’t actually know.

          2). Don’t really understand.

          3). Am not a subject matter expert.

          4). Have no idea where to start.

          In under 20 minutes. I’ve worked on code bases for months where I would still not be half as productive.

          1. 4

            This comment was more useful than the original post and the fact that, as of this writing, the comment has more upvotes than the story, I think others agree. Thank you for this!

            1. 3

              Thanks! Glad it was useful to someone!

            2. 3

              I agree mostly but the argument for ConTeXt does not convince me. ConTeXt to LaTeX is pretty much Ruby to Perl. Neither provides a better solution. As I have most of the scripts in Perl already, I am not going to touch Ruby. I will wait until Python comes along and all the cool kids started writing all kinds of scripts, so much so that I can pip anything and forgets about cpan. Until then, I’ll stick to LaTeX.

              1. 1

                Yes, the greater your investment in LaTeX, the more difficult a switch would be. I’m fortunate in that all of my projects are pretty independent and I don’t do them very frequently, so I could afford to make these deep dives for my own edification. Most people who are using LaTeX day-to-day are compelled to do so by journals or institutional requirements and don’t have the luxury to switch to ConTeXt. Even if you have the option, I can respect not wanting to deal with a new system, especially if you have a lot of built-up knowledge about how to solve your problems with LaTeX packages. The package ecosystem everywhere else is much smaller.

            3. 11

              As far as I can see this is just a single blog post with some critique of TeX/LaTeX and a wishlist.

              There’s no code, no further discussion of architecture etc?

              1. 4

                The other blogs on the same page are related, but this project does seem to be in the very early stages.

              2. 4

                ConTeXt doesn’t seem to be included in this analysis of TeX engines & macro packages?

                1. 5

                  I agree that in principle ConTeXt is probably the best situated of the various TeXans. But TFA doesn’t really give any analysis to speak of of anything aside from TeX and LaTeX, except to say ‘fragmentation bad’.

                2. 4

                  Him; I’m not sure I understand the article well, but I’d recommend checking out SILE (https://sile-typesetter.org) for anyone interested in a modern alternative inspired by LaTeX. In a true hacker fashion, SILE brutally reuses a carefully selected set of its predecessor’s subcomponents, gluing them in a new way to create something completely different yet familiar. Caveat: math support is not there yet; it’s open to contributions, and I tried to do that, got quite far, but run out of juice at the ever so crucial finish stage (a.k.a. “the other 80%”).

                  1. 6

                    I came here to highlight SILE. SILE starts from identifying two deficiencies in the TeX ecosystem:

                    • The TeX VM is basically a Turing Machine. That is an awful programming model for humans to use. Writing extensions to TeX is painful. SILE fixes this by starting with a real language (Lua) and implements the core typesetting system in the same language as the extensions so packages are first-class components.
                    • TeX made a load of compromises for practicality that are not . For example, TeX has a really nice dynamic programming approach to laying out words in a paragraph. The paper that presents this explains that the same approach could be used for laying out paragraphs on a page, but for a book-length document this might need as much as a megabyte of memory for the intermediate state. Sile implements the algorithms that the TeX designers wanted to implement but couldn’t due to resource constraints.

                    There are some trivial things that TeX can’t express. For example, most style guides say that every figure should go after the first reference to it, but the eager layout in TeX makes this impossible. It’s very easy in Sile, which can do arbitrary backtracking. One of the original use cases for Sile was to typeset bibles, where you need to lay out text on pairs of pages together to minimise bleed-through on thin paper.