Hi, Typst dev here. You understand correctly. The GUI is closed source and the compiler is open source. Some of the web app’s IDE-like features (e.g. autocomplete) are part of the open source library though. Right now, everything is completely free, but at some point in the future the web app will have paid features.
Typst is very exciting - it seems like the first meaningful LaTeX competitor in a long time.
And it seems they’ve just implemented footnotes, which was the main blocker for me using it for schoolwork! I’ll probably give it another shot in September.
I really like SILE and I wish I could use it for papers, but there are some blockers:
Conferences all provide LaTeX (and, often, Word) templates. If you use these, you won’t be rejected for failing to conform to their style, if you have to recreate something like the SIGPLAN style in SILE then that’s a lot of effort.
Last time I looked, it was still missing equivalents of math mode and bibtex, which are essential for any scientific publication.
LaTeX has a load of packages that do everything I want. Generating tables and plots from data? Pgfplots and pgfplotstable are there for me. Typesetting code? Listings is fine there are also some newer things that wrap external parsers. Diagrams, pseudocode, whatever all have some LaTeX package.
SILE dev here. Some non-exhaustive notes to your three points:
True, the demand from conferences, universities, and journals to use X template exactly is a hard one. Note this also affects typst just as much as sile. That being said it is possible to write classes for SILE and for the most part shouldn’t even be too hard to replicate the layout and rules for many documents to come out the same as long as the product they want is a PDF and not the LaTeX or Word sources. I’m personally in the book business and not so much academia, but I’m happy to help package devs if anybody wants to tackle setting up a class to mimic some style guide.
It might be a while since you looked, SILE supports math and bibtex. The math support is less refined than in Typst or LaTeX, but it covers most of MathML. Typst came out with focus on math first, for SILE is was contributed later. On the flip side Typst has very rudimentary support for many other typesetting features like frames, footnotes, RTL and TTB writing directions, hyphenation and special features languages and scripts, etc., while SILE focused on those out of the gate. Your needs may vary and a different solution might be better for different projects.
True, there is a huge depth to the LaTeX ecosystem. There are only a few dozen core SILE packages and another handful of 3rd party ones. That being said almost all the things you mention are doable, and parsing other inputs (such as diagraming) and embedding the output is way easier in SILE than LaTeX. One major difference between SILE and Typst is that Typst strictly limits what 3rd party packages/templates/functions can do, while SILE provides a Lua interface with 100% access to extend or tinker with it’s own internals as well as reach out to the system and run other programs to generate output. SILE also has a package management solution for 3rd party packages that is a bit more robust than Typst’s current allowance of mostly copy/pasting code into your project.
That being said it is possible to write classes for SILE and for the most part shouldn’t even be too hard to replicate the layout and rules for many documents to come out the same as long as the product they want is a PDF and not the LaTeX or Word sources. I’m personally in the book business and not so much academia, but I’m happy to help package devs if anybody wants to tackle setting up a class to mimic some style guide.
I wonder if there’s a volunteer at the ACM who would be interested in at least making their templates work with SILE. Most conferences I care about use either an ACM or IEEE style.
It might be a while since you looked, SILE supports math and bibtex.
It is, I looked in quite a bit of detail after having breakfast with Simon at FOSDEM many years ago, but have only periodically checked for progress. I saw some progress recently on an issue that I filed a few years back about documenting the APIs. Great news. How does BibTeX support work? BibTeX can include arbitrary TeX, which makes it painful, but I tend not to do that because I also want my BibTeX to work with things like Jekyll Scholar, so more limited support is probably fine for me.
There are only a few dozen core SILE packages and another handful of 3rd party ones. That being said almost all the things you mention are doable, and parsing other inputs (such as diagraming) and embedding the output is way easier in SILE than LaTeX
Things like TikZ and pfgplots[data] are useful precisely because I don’t need an external tool. In particular, the main reason that I use TikZ is that the fonts in my diagrams match those in my document. Most of the TeX fonts are also available as OpenType, so you can do this, but then if you decide to reduce the text size by 1pt you have to go and make the matching change in all of the figures. To give another concrete example, I have made very heavy use of a LaTeX package that draws bit patterns in machine words for describing instruction encodings. All of this can be done in SILE (probably in considerably less than half the effort of doing it in TeX and with better performance), but someone else has done it in LaTeX. I really hope that the SILE ecosystem starts to grow this kind of thing.
If there is anybody that wants to give it a shot with me I’d be happy to spin up a class for ACM/IEEE/whatever popular style guide is out there and facilitate the process of getting everything implemented as close a match as possible. If this interests anybody pop open a new issue with a link to the style guide itself and a sample input/output document (in LaTeX/PDF presumably) and I’ll fire up the boilerplate for a new class package and get he ball rolling. Obviously the simpler/more flexible the style guide we start with the better, and the simpler the PoC input we want to start with the better.
Obviously RAW TeX in BibTeX doesn’t work great, although since “turn about is fair play” we can process RAW SIL input (or XML) in BibTeX content, although our ability to process it as RAW Markdown is probably more useful. On other words it can be reasonably adapted for any input format(s), just like SILE itself can take XML, SIL, Markdown, or other input formats.
And yes, your comments about “this is hard in LaTeX but somebody already did it” are pretty much true. I still use LaTeX some myself just because some projects are already all setup to work with it, so the momentum keeps the ball rolling. I hate having to tweak things because the internals are painful, but for the stuff that already works out of the box it’s hard to argue with nice output. I too hope over time the ease of implementing new things in SILE with access to a full fledged language ecosystem (several actually since you can easily write C, C++, Rust, and execute it as a Lua package as well as shim in/shell out Python and others with bindings too) will allow the 3rd party ecosystem for SILE to thrive too.
Is there a good AsciiDoc to SILE flow? I used LaTeX for my prior books but I’m looking at AsciiDoc for the next one, since it lets me embed semantic markup that can flow through to things that I can style with CSS for the ePub version (I wrote my own TeX -> HTML converter for my last couple of books because all of the LaTeX tooling I tried made a mess of it), but I’d love to be able to use SILE for PDF versions.
That isn’t a flow I’ve ever done in production end to end, but there are several paths you could take. AsciiDoc → DocBook via Pandoc would be one, then directly typset using SILE’s DocBook support (XML with a set of style rules for tags basically). Another path would be again using Pandoc to convert to Markdown, then use the markdown.sile inputter. A third way might be to use CaSILE which bundles a Pandoc fork that includes a SILE writer. The extra versatility in this is it allows you to embed raw SIL commands in your input (works for Markdown, not sure how AsciiDoc would handle this but I think there is a way). Feel free to open a discussion on SILE for either of the former or a feature request issue for CaSILE if they interest you. Any of the above should take just a little bit of fiddling to work out the right commands. If you want both PDF and ePUB output from plain text sources in any format CaSILE is what I would reach for (that’s what I made it for).
I tried AsciiDoc -> DocBook -> SILE workflow recently. My conclusion is:
this is theoretically and practically supported workflows. SILE has first-class support for ingesting DocBook
this seems like a “do the right thing” principled approach, rather than just a pile of hacks
at the same time, the actual state of support for various DocBook elements is poor: https://github.com/sile-typesetter/sile/issues/1338. If you want to use it, be prepared to contribute a good half of it upstream yourself
Realistically if you want to typeset DocBook you probably are going to want to style all the tags yourself. About have of them have default styling and the other are pass-throughs just to get the content to the page, but would need styling for use. The default styling though is almost certainly now what you want for any book project.
Thanks! Does this flow properly preserve custom elements in AsciiDoc through DocBook to places where SILE can pick them up? For example, in my last book I wanted to make sure that the code examples all worked and so each one was in a file that could be compiled and tested and referenced from the LaTeX and so I would want to add some handling in SILE to grab the right lines from the file (identified by delimiters in comments), pares the file with something external, and then feed that as text with particular styling into the SILE pipeline.
I don’t know, but I think this should work, yeah. I think what you are asking belongs to AsciiDoc -> DocBook step (I think it would be on the asciidoctor processor to do read the source code, extract specific lines, and emit DocBook with text already “pre-processed”), and, given that AsciiDoc is essentially a different syntax for DocBook, this step should work perfectly in practice.
In any case, on SILE side you use lua to do whatever (as an aside, after wondering “why typist is a whole programming language” and looking closer at SILE, I realize that this is because typesetting itself is essentially imperative. https://sile-typesetter.org/manual/sile-0.14.9.pdf#page123 Is a good example).
Fun context: historically, I advocated for the use of AsciiDoctor. Since then, I personally switched to Djot (which isn’t yet production ready), and I was looking for a great way to convert Djot to PDF. I looked at “Djot -> DocBook -> SILE” route, and, while I was ready to do the first step from-scratch myself, I don’t have enough domain knowledge to help polish the second step to “just works” stage. So I went “Djot -> HTML -> Chromium -> Print to PDF” route, with its horrible typesetting.
I believe markdown.sile also supports Djot directly as an inputter, that gives you Djot -> SILE. If it isn’t that package Djot support is in another package by the same author.
Thanks for the tip about markdown.sile! Will try that out once SILE works again for me with nixos (Package ‘lua5.4-lua-zlib-1.2-2’ is marked as broken, if usng lua5.4, and Package ‘lua5.3-vstruct-2.1.1-1’ is marked as broken for 5.3 TT)
Even if recent builds of those Lua packages are broken in Nix, you should still be able to run sile at the last package version of SILE because the dependencies by using locked dependency versions from the time of packaging. I can’t remember the incantation for that inside NixOS but I know there is one. You can also run the Flake version which has it’s own lockfile with dependencies at known working versions. For example nix run github:sile-typesetter/sile runs the latest flake version, but you can also run any previous version since we introduced the Flake complete with whatever dependencies they had at the time.
Feel free to tag me in Lua package related issues in nixpkgs if they affect SILE.
I wonder if there’s a volunteer at the ACM who would be interested in at least making their templates work with SILE.
I think you would need more than that, since ACM is moving to a publication workflow that ingests article source, not author-submitted PDFs. Not every conference (yet?) requires it, but some big ones do. The end result is that LaTeX or Word are really the only two options. I have met a few people who draft articles primarily in Markdown, and then have a home-rolled converter to ACM-compliant LaTeX. But that only works because they use a small subset of Markdown features that can be one-to-one translated (like sections).
Yes. I caveated my suggestion/offer above noting it will only help at this point for submission systems that just need the final output product, not the source.
This is a culture problem, not just a tech one though. My hope is that eventually the tooling to build/convert is standardized just enough that folks can write in their choice of LaTeX/Typst/SILE and submit the source repository, and the other end could process any of those, say using a standardized CI workflow with options for different typesetters. The workflow and output formats could still be standardized while giving some tooling choices on how to get there.
Wondering if you’d mind answering some semi-tangential questions of mine regarding SILE/TeX/typesetting.
How hard is it to get the “core” of TeX right? From the FAQ, it seems that SILE has feature parity (or better features) than TeX, but as noted is missing the ecosystem.
How hard would it be to transpile or interpret existing TeX packages? How does this compare with the amount of effort that would be required to rewrite them?
That depends a lot of what you think of as “core” TeX. I’m assuming you don’t understand the actual boundaries of TeX vs. the LaTeX/XeLaTeX/LuaTeX/ConTeXt ecosystems. I assume you are talking about basic typesetting algorithms such as line breaking or leading. This doesn’t match up cleanly at all with what core TeX actually is, but we’ll go with the basic algorithms. These took a lot of thought to get right, but they are quite easy to clone. They have been ported to dozen of languages. SILE itself has almost exact ports of a few factions (including step numbers in source comments from the original). The issue really isn’t implementing any one part, the issue is coming up with a system where they can all get along. Taking a string and wrapping it at a certain length with the line breaker isn’t too hard. In a naive box, that would be the end of things, and that’s where core TeX kind of leaves you to. LaTeX for all it’s huge ecosystem just doesn’t handle some page layouts though. SILE has taken a long time to work out how to make those core bits work as expected when you start chopping the box up into variable size frames that grow when you have sidebar insertions or mix and match with other layout adjustments. It also gets complicated when you start dealing with things outside of English and a few other Latin script based languages. Suddenly some of the core TeX stuff falls apart and you realize you need much more involved interactions between different parts of the chain to get things right. Transpiling TeX is possible, but it still leaves you with all the idiosyncrasies and limitations of TeX. There is a pure Python implementation called yex for example, but it doesn’t get you much farther (or even as far) as TeX itself.
Ground-up rethinks like Typst (Rust/custom) or XTS (Go) or SILE (Lua/Rust) are important because there is lots more wrong with the fundamental workings of TeX that just the language the typesetter is implemented in. Sure that makes it extra hard to fix these days, but as far as typesetting engines go none of the modern alternatives are really winners because of the language the engine is written in, at the end of the day they mostly stand or fall based on what they can accomplish.
Thanks for the detailed response, that helps clarify my understanding of the state of things. And you’re right, I don’t know jack about the ecosystem. Looking at the Typst’s description of how XeLaTeX is compiled was mind boggling to me.
To explain my questions: I’ve long desired something that acts like TeX but doesn’t look like it. For example, Typst with TeX-style math and - importantly - access to most TeX packages. I don’t use TeX for anything complicated or (that) precise. Like 80% of the documents I write are either simple prose or (often reasonably simple) math. I really would love something that can hide the complexity of the “fiddly bits” of TeX, or at least wrap it in a DSL I can actually interact with. I can see now, though, how my opinion as a user of TeX is naive to someone who knows about how TeX works. I do think that if someone can make “Typst (i.e. markdown-like front end + a way to deal with the fiddly bits that doesn’t make me so, so sad) with backward compatability with the TeX ecosystem” that would be awesome, but I suppose I understand now that this is difficult if not impossible.
SILE might be exactly what you want. Simon started with the original TeX papers and implemented not just the algorithms that TeX uses but also the ones that TeX wanted to use but were infeasible. For example, TeX has a really nice dynamic processing algorithm for line breaking that takes a whole paragraph, assigns badness scores to each possible breaking position, and finds a solution that minimises badness. The paper describing this points out that the same approach could be used for laying out paragraphs in a document but that, for large documents, this would need a megabyte of memory and so is infeasible. SILE implements that version, because it’s now a trivial amount of memory (even with the overhead of using a Lua table rather than a C array) is trivial.
This means that there are some important things that you can express in SILE but not in TeX. For example, most publishers have a rule that a figure must not appear before the first reference to it. TeX can’t express that because it places the figure using a greedy algorithm. SILE can assign a maximum badness value to any possible placement of the figure before it’s first reference and then have it placed nicely.
At the same time, SILE is written in Lua, whereas TeX is a horrible Turing machine that is the reason that I don’t listen to anything Knuth has to say about software engineering (while having nothing but respect for him as an expert on algorithms). This means that humans have a chance of being able to extend SILE.
Simon chose Lua specifically to expose the entire pipeline to users and allow anything to be patched (well, actually, he chose JavaScript for this reason and was persuaded to rewrite it in Lua and have the same benefits). This means that it should be easy to embed in other things and extend arbitrarily.
Have you looked at KeenWrite? it’s my free, open-source, cross-platform, desktop text editor that invokes ConTeXt for typesetting. The editor uses KeenType for its real-time mathematics preview.
For me the “concurrent” would be Overleaf while you would still be in LaTeX ecosystem vs the mix of Asciidoc and custom operators that Typst seems to propose. Maybe this could be lower the barrier for students to be able to produce better looking papers (and stop some of them to write their table of contents by hand in Word :’) ).
Speaking of live preview, I wrote my own tool for that and for error report: https://github.com/let-def/texpresso .
This is not a full solution like Typst, rather an easy-to-maintain hack on top of XeTeX… A small quality-of-life improvement for people who have no choice but to use LaTeX at the moment.
That being said, I am glad people are working toward better solutions for typesetting, the current state is really unsatisfying.
Technically, it is a custom DVI/XDV renderer driving a patched version of (Xe)TeX engine that keeps a few forks around so that when the input change, it can resume a TeX instance that has already processed the unchanged prefix. So it is unix only since it fundamentally relies on fork. In practice, only the page being edited has to be rendered, that can happen in a few dozen milliseconds.
For the UI, I wrote a minor emacs mode that integrates transparently with the rest of emacs ecosystem.
Maybe its the IKEA effect, but that makes LaTeX much more pleasant to me (and milliseconds iteration cycle helps a lot when dealing with cryptic error messages).
Wow this is cool! And also way cooler than my own live preview hacks for LaTeX from back in the day. I would love to develop something like this for SILE. Especially with the upcoming major release with a Rust wrapper we could easily cache not just the AST but all our processing work up to that point, then diff an incoming AST with changes to find where they start, then feed the state from a previous SILE run back into the engine to pick up where the new changes start. This wouldn’t be quite as principled an approach as Typst has (they did a good job on incremental compilation), but it would be oodles faster than laying out a couple hundred pages of a book from scratch for a change near the end.
Am I understanding correctly that the whole GUI is closed source, only the CLI is open source? I’m not criticizing, I’m asking :)
Hi, Typst dev here. You understand correctly. The GUI is closed source and the compiler is open source. Some of the web app’s IDE-like features (e.g. autocomplete) are part of the open source library though. Right now, everything is completely free, but at some point in the future the web app will have paid features.
Why are all of their communications options closed source? GitHub + Discord? smh
Typst is very exciting - it seems like the first meaningful LaTeX competitor in a long time.
And it seems they’ve just implemented footnotes, which was the main blocker for me using it for schoolwork! I’ll probably give it another shot in September.
Have you considered sile? https://sile-typesetter.org/what-is-sile/
I really like SILE and I wish I could use it for papers, but there are some blockers:
SILE dev here. Some non-exhaustive notes to your three points:
True, the demand from conferences, universities, and journals to use X template exactly is a hard one. Note this also affects
typstjust as much assile. That being said it is possible to write classes for SILE and for the most part shouldn’t even be too hard to replicate the layout and rules for many documents to come out the same as long as the product they want is a PDF and not the LaTeX or Word sources. I’m personally in the book business and not so much academia, but I’m happy to help package devs if anybody wants to tackle setting up a class to mimic some style guide.It might be a while since you looked, SILE supports math and bibtex. The math support is less refined than in Typst or LaTeX, but it covers most of MathML. Typst came out with focus on math first, for SILE is was contributed later. On the flip side Typst has very rudimentary support for many other typesetting features like frames, footnotes, RTL and TTB writing directions, hyphenation and special features languages and scripts, etc., while SILE focused on those out of the gate. Your needs may vary and a different solution might be better for different projects.
True, there is a huge depth to the LaTeX ecosystem. There are only a few dozen core SILE packages and another handful of 3rd party ones. That being said almost all the things you mention are doable, and parsing other inputs (such as diagraming) and embedding the output is way easier in SILE than LaTeX. One major difference between SILE and Typst is that Typst strictly limits what 3rd party packages/templates/functions can do, while SILE provides a Lua interface with 100% access to extend or tinker with it’s own internals as well as reach out to the system and run other programs to generate output. SILE also has a package management solution for 3rd party packages that is a bit more robust than Typst’s current allowance of mostly copy/pasting code into your project.
I wonder if there’s a volunteer at the ACM who would be interested in at least making their templates work with SILE. Most conferences I care about use either an ACM or IEEE style.
It is, I looked in quite a bit of detail after having breakfast with Simon at FOSDEM many years ago, but have only periodically checked for progress. I saw some progress recently on an issue that I filed a few years back about documenting the APIs. Great news. How does BibTeX support work? BibTeX can include arbitrary TeX, which makes it painful, but I tend not to do that because I also want my BibTeX to work with things like Jekyll Scholar, so more limited support is probably fine for me.
Things like TikZ and pfgplots[data] are useful precisely because I don’t need an external tool. In particular, the main reason that I use TikZ is that the fonts in my diagrams match those in my document. Most of the TeX fonts are also available as OpenType, so you can do this, but then if you decide to reduce the text size by 1pt you have to go and make the matching change in all of the figures. To give another concrete example, I have made very heavy use of a LaTeX package that draws bit patterns in machine words for describing instruction encodings. All of this can be done in SILE (probably in considerably less than half the effort of doing it in TeX and with better performance), but someone else has done it in LaTeX. I really hope that the SILE ecosystem starts to grow this kind of thing.
If there is anybody that wants to give it a shot with me I’d be happy to spin up a class for ACM/IEEE/whatever popular style guide is out there and facilitate the process of getting everything implemented as close a match as possible. If this interests anybody pop open a new issue with a link to the style guide itself and a sample input/output document (in LaTeX/PDF presumably) and I’ll fire up the boilerplate for a new class package and get he ball rolling. Obviously the simpler/more flexible the style guide we start with the better, and the simpler the PoC input we want to start with the better.
Obviously RAW TeX in BibTeX doesn’t work great, although since “turn about is fair play” we can process RAW SIL input (or XML) in BibTeX content, although our ability to process it as RAW Markdown is probably more useful. On other words it can be reasonably adapted for any input format(s), just like SILE itself can take XML, SIL, Markdown, or other input formats.
And yes, your comments about “this is hard in LaTeX but somebody already did it” are pretty much true. I still use LaTeX some myself just because some projects are already all setup to work with it, so the momentum keeps the ball rolling. I hate having to tweak things because the internals are painful, but for the stuff that already works out of the box it’s hard to argue with nice output. I too hope over time the ease of implementing new things in SILE with access to a full fledged language ecosystem (several actually since you can easily write C, C++, Rust, and execute it as a Lua package as well as shim in/shell out Python and others with bindings too) will allow the 3rd party ecosystem for SILE to thrive too.
Is there a good AsciiDoc to SILE flow? I used LaTeX for my prior books but I’m looking at AsciiDoc for the next one, since it lets me embed semantic markup that can flow through to things that I can style with CSS for the ePub version (I wrote my own TeX -> HTML converter for my last couple of books because all of the LaTeX tooling I tried made a mess of it), but I’d love to be able to use SILE for PDF versions.
That isn’t a flow I’ve ever done in production end to end, but there are several paths you could take. AsciiDoc → DocBook via Pandoc would be one, then directly typset using SILE’s DocBook support (XML with a set of style rules for tags basically). Another path would be again using Pandoc to convert to Markdown, then use the
markdown.sileinputter. A third way might be to use CaSILE which bundles a Pandoc fork that includes a SILE writer. The extra versatility in this is it allows you to embed raw SIL commands in your input (works for Markdown, not sure how AsciiDoc would handle this but I think there is a way). Feel free to open a discussion on SILE for either of the former or a feature request issue for CaSILE if they interest you. Any of the above should take just a little bit of fiddling to work out the right commands. If you want both PDF and ePUB output from plain text sources in any format CaSILE is what I would reach for (that’s what I made it for).I tried AsciiDoc -> DocBook -> SILE workflow recently. My conclusion is:
Realistically if you want to typeset DocBook you probably are going to want to style all the tags yourself. About have of them have default styling and the other are pass-throughs just to get the content to the page, but would need styling for use. The default styling though is almost certainly now what you want for any book project.
Thanks! Does this flow properly preserve custom elements in AsciiDoc through DocBook to places where SILE can pick them up? For example, in my last book I wanted to make sure that the code examples all worked and so each one was in a file that could be compiled and tested and referenced from the LaTeX and so I would want to add some handling in SILE to grab the right lines from the file (identified by delimiters in comments), pares the file with something external, and then feed that as text with particular styling into the SILE pipeline.
I don’t know, but I think this should work, yeah. I think what you are asking belongs to AsciiDoc -> DocBook step (I think it would be on the asciidoctor processor to do read the source code, extract specific lines, and emit DocBook with text already “pre-processed”), and, given that AsciiDoc is essentially a different syntax for DocBook, this step should work perfectly in practice.
In any case, on SILE side you use lua to do whatever (as an aside, after wondering “why typist is a whole programming language” and looking closer at SILE, I realize that this is because typesetting itself is essentially imperative. https://sile-typesetter.org/manual/sile-0.14.9.pdf#page123 Is a good example).
Fun context: historically, I advocated for the use of AsciiDoctor. Since then, I personally switched to Djot (which isn’t yet production ready), and I was looking for a great way to convert Djot to PDF. I looked at “Djot -> DocBook -> SILE” route, and, while I was ready to do the first step from-scratch myself, I don’t have enough domain knowledge to help polish the second step to “just works” stage. So I went “Djot -> HTML -> Chromium -> Print to PDF” route, with its horrible typesetting.
I believe markdown.sile also supports Djot directly as an inputter, that gives you Djot -> SILE. If it isn’t that package Djot support is in another package by the same author.
Thanks for the tip about
markdown.sile! Will try that out once SILE works again for me with nixos (Package ‘lua5.4-lua-zlib-1.2-2’ is marked as broken,if usng lua5.4, andPackage ‘lua5.3-vstruct-2.1.1-1’ is marked as brokenfor 5.3 TT)Even if recent builds of those Lua packages are broken in Nix, you should still be able to run
sileat the last package version of SILE because the dependencies by using locked dependency versions from the time of packaging. I can’t remember the incantation for that inside NixOS but I know there is one. You can also run the Flake version which has it’s own lockfile with dependencies at known working versions. For examplenix run github:sile-typesetter/sileruns the latest flake version, but you can also run any previous version since we introduced the Flake complete with whatever dependencies they had at the time.Feel free to tag me in Lua package related issues in nixpkgs if they affect SILE.
I think you would need more than that, since ACM is moving to a publication workflow that ingests article source, not author-submitted PDFs. Not every conference (yet?) requires it, but some big ones do. The end result is that LaTeX or Word are really the only two options. I have met a few people who draft articles primarily in Markdown, and then have a home-rolled converter to ACM-compliant LaTeX. But that only works because they use a small subset of Markdown features that can be one-to-one translated (like sections).
Yes. I caveated my suggestion/offer above noting it will only help at this point for submission systems that just need the final output product, not the source.
This is a culture problem, not just a tech one though. My hope is that eventually the tooling to build/convert is standardized just enough that folks can write in their choice of LaTeX/Typst/SILE and submit the source repository, and the other end could process any of those, say using a standardized CI workflow with options for different typesetters. The workflow and output formats could still be standardized while giving some tooling choices on how to get there.
Wondering if you’d mind answering some semi-tangential questions of mine regarding SILE/TeX/typesetting.
That depends a lot of what you think of as “core” TeX. I’m assuming you don’t understand the actual boundaries of TeX vs. the LaTeX/XeLaTeX/LuaTeX/ConTeXt ecosystems. I assume you are talking about basic typesetting algorithms such as line breaking or leading. This doesn’t match up cleanly at all with what core TeX actually is, but we’ll go with the basic algorithms. These took a lot of thought to get right, but they are quite easy to clone. They have been ported to dozen of languages. SILE itself has almost exact ports of a few factions (including step numbers in source comments from the original). The issue really isn’t implementing any one part, the issue is coming up with a system where they can all get along. Taking a string and wrapping it at a certain length with the line breaker isn’t too hard. In a naive box, that would be the end of things, and that’s where core TeX kind of leaves you to. LaTeX for all it’s huge ecosystem just doesn’t handle some page layouts though. SILE has taken a long time to work out how to make those core bits work as expected when you start chopping the box up into variable size frames that grow when you have sidebar insertions or mix and match with other layout adjustments. It also gets complicated when you start dealing with things outside of English and a few other Latin script based languages. Suddenly some of the core TeX stuff falls apart and you realize you need much more involved interactions between different parts of the chain to get things right. Transpiling TeX is possible, but it still leaves you with all the idiosyncrasies and limitations of TeX. There is a pure Python implementation called
yexfor example, but it doesn’t get you much farther (or even as far) as TeX itself.Ground-up rethinks like Typst (Rust/custom) or XTS (Go) or SILE (Lua/Rust) are important because there is lots more wrong with the fundamental workings of TeX that just the language the typesetter is implemented in. Sure that makes it extra hard to fix these days, but as far as typesetting engines go none of the modern alternatives are really winners because of the language the engine is written in, at the end of the day they mostly stand or fall based on what they can accomplish.
Thanks for the detailed response, that helps clarify my understanding of the state of things. And you’re right, I don’t know jack about the ecosystem. Looking at the Typst’s description of how XeLaTeX is compiled was mind boggling to me.
To explain my questions: I’ve long desired something that acts like TeX but doesn’t look like it. For example, Typst with TeX-style math and - importantly - access to most TeX packages. I don’t use TeX for anything complicated or (that) precise. Like 80% of the documents I write are either simple prose or (often reasonably simple) math. I really would love something that can hide the complexity of the “fiddly bits” of TeX, or at least wrap it in a DSL I can actually interact with. I can see now, though, how my opinion as a user of TeX is naive to someone who knows about how TeX works. I do think that if someone can make “Typst (i.e. markdown-like front end + a way to deal with the fiddly bits that doesn’t make me so, so sad) with backward compatability with the TeX ecosystem” that would be awesome, but I suppose I understand now that this is difficult if not impossible.
SILE might be exactly what you want. Simon started with the original TeX papers and implemented not just the algorithms that TeX uses but also the ones that TeX wanted to use but were infeasible. For example, TeX has a really nice dynamic processing algorithm for line breaking that takes a whole paragraph, assigns badness scores to each possible breaking position, and finds a solution that minimises badness. The paper describing this points out that the same approach could be used for laying out paragraphs in a document but that, for large documents, this would need a megabyte of memory and so is infeasible. SILE implements that version, because it’s now a trivial amount of memory (even with the overhead of using a Lua table rather than a C array) is trivial.
This means that there are some important things that you can express in SILE but not in TeX. For example, most publishers have a rule that a figure must not appear before the first reference to it. TeX can’t express that because it places the figure using a greedy algorithm. SILE can assign a maximum badness value to any possible placement of the figure before it’s first reference and then have it placed nicely.
At the same time, SILE is written in Lua, whereas TeX is a horrible Turing machine that is the reason that I don’t listen to anything Knuth has to say about software engineering (while having nothing but respect for him as an expert on algorithms). This means that humans have a chance of being able to extend SILE.
Simon chose Lua specifically to expose the entire pipeline to users and allow anything to be patched (well, actually, he chose JavaScript for this reason and was persuaded to rewrite it in Lua and have the same benefits). This means that it should be easy to embed in other things and extend arbitrarily.
Have you looked at KeenWrite? it’s my free, open-source, cross-platform, desktop text editor that invokes ConTeXt for typesetting. The editor uses KeenType for its real-time mathematics preview.
See the video tutorials for details.
I wish this had come out when I was in university. It looks so cool but I don’t really have a reason to use it right now.
For me the “concurrent” would be Overleaf while you would still be in LaTeX ecosystem vs the mix of Asciidoc and custom operators that Typst seems to propose. Maybe this could be lower the barrier for students to be able to produce better looking papers (and stop some of them to write their table of contents by hand in Word :’) ).
Right now, GPT is taking care of latex for me.
Good for you, but GPT doesn’t help with live preview or collaboration.
I still find it hard to work with latex though. I downloaded typst and boy this is something….
Speaking of live preview, I wrote my own tool for that and for error report: https://github.com/let-def/texpresso . This is not a full solution like Typst, rather an easy-to-maintain hack on top of XeTeX… A small quality-of-life improvement for people who have no choice but to use LaTeX at the moment. That being said, I am glad people are working toward better solutions for typesetting, the current state is really unsatisfying.
Technically, it is a custom DVI/XDV renderer driving a patched version of (Xe)TeX engine that keeps a few forks around so that when the input change, it can resume a TeX instance that has already processed the unchanged prefix. So it is unix only since it fundamentally relies on fork. In practice, only the page being edited has to be rendered, that can happen in a few dozen milliseconds. For the UI, I wrote a minor emacs mode that integrates transparently with the rest of emacs ecosystem. Maybe its the IKEA effect, but that makes LaTeX much more pleasant to me (and milliseconds iteration cycle helps a lot when dealing with cryptic error messages).
Wow this is cool! And also way cooler than my own live preview hacks for LaTeX from back in the day. I would love to develop something like this for SILE. Especially with the upcoming major release with a Rust wrapper we could easily cache not just the AST but all our processing work up to that point, then diff an incoming AST with changes to find where they start, then feed the state from a previous SILE run back into the engine to pick up where the new changes start. This wouldn’t be quite as principled an approach as Typst has (they did a good job on incremental compilation), but it would be oodles faster than laying out a couple hundred pages of a book from scratch for a change near the end.