The way this works is that the language server runs the compiler, parses its output, transforms it to the JSON format according to the LSP protocol, sends that other the wire to the editor, which parses the LSP json into editors internal representation for diagnostics.
In this picture, the language server is completely unnecessary. It’s just a lot of complexity overhead (and some CPU overhead). What should be happening is that the editor should itself call the build system directly, and then parse out its diagnostics.
What this needs though, is a cross-compiler standard for machine readable diagnostics. There’s no such standard yet, so language servers which can’t actually provide a full live set of diagnostics (including rust-analyzer) have to play the middleman, even though architecturally that’s a wrong approach.
If you forget about the process boundaries for a sec, then the language server is just the component that is responsible for mapping the toolchain’s output to a cross-compiler format the editor understands. The inefficiency of involving an additional process here doesn’t feel so bad to me, especially in that it means the editor gets crash isolation from that toolchain.
In a hypothetical ideal world the language toolchain could stay memory-resident for performance reasons, in which case having it live in a separate long-lived process instead of an editor-spawned toolchain is also necessary. (And this is exactly how the TypeScript toolchain works and why it is so fast…) From that perspective, having the editor only interact with an LSP vs having additional toolchain-spawning logic makes sense to me.
But, at the end of the day (well, at least until we go to the distributed builds territory), you are going to call into a proper build system, which would compile project as a whole.
I think it’s useful to model these two sorts of diagnostic sources separately: for smaller, newer languages it allows you to get a decent in-editor error experience with little effort (yay flymake), for larger projects, it solves the impedance mismatch between what you language server thinks the project structure is and your build system’s ground truth.
bad to me, especially in that it means the editor gets crash isolation from that toolchain.
I don’t think this gets you more crash isolation? It’s an “external process” either way.
But this is related to the actual reason why I think asking language server to manage build system leads to worse UX. Broadly, spawning processes from within a language server is scary, because processes can misbehave: your compiler might eat a Zip bomb or your package manager can get stuck in a DNS. And, from within the language server, you don’t really have any way to inform the user that “hey, this thing have been running for 10 minutes and it looks dead”.
In contrast, editors typically already have some sort of UI for managing a list of currently running external processes. So, if it is the editor which manages build system invocation, it can easily draw the “cancel” button right next to the in-progress output to allow the user to kill any runaway builds.
Some notes on the implementation side of things:
The way this works is that the language server runs the compiler, parses its output, transforms it to the JSON format according to the LSP protocol, sends that other the wire to the editor, which parses the LSP json into editors internal representation for diagnostics.
In this picture, the language server is completely unnecessary. It’s just a lot of complexity overhead (and some CPU overhead). What should be happening is that the editor should itself call the build system directly, and then parse out its diagnostics.
What this needs though, is a cross-compiler standard for machine readable diagnostics. There’s no such standard yet, so language servers which can’t actually provide a full live set of diagnostics (including rust-analyzer) have to play the middleman, even though architecturally that’s a wrong approach.
SARIF might serve a role as that kind of standard.
If you forget about the process boundaries for a sec, then the language server is just the component that is responsible for mapping the toolchain’s output to a cross-compiler format the editor understands. The inefficiency of involving an additional process here doesn’t feel so bad to me, especially in that it means the editor gets crash isolation from that toolchain.
In a hypothetical ideal world the language toolchain could stay memory-resident for performance reasons, in which case having it live in a separate long-lived process instead of an editor-spawned toolchain is also necessary. (And this is exactly how the TypeScript toolchain works and why it is so fast…) From that perspective, having the editor only interact with an LSP vs having additional toolchain-spawning logic makes sense to me.
Not quiet! The language server has a time dimension — it manages the live set of diagnostics changing over time. This is useful if you want to provide incremental diagnostics for subset of the files (and “subset of files” does a lot of performance heavy lifting here, https://stackoverflow.com/questions/55201424/how-to-get-vscode-to-show-typescript-errors-for-files-not-open-in-the-editor)
But, at the end of the day (well, at least until we go to the distributed builds territory), you are going to call into a proper build system, which would compile project as a whole.
I think it’s useful to model these two sorts of diagnostic sources separately: for smaller, newer languages it allows you to get a decent in-editor error experience with little effort (yay flymake), for larger projects, it solves the impedance mismatch between what you language server thinks the project structure is and your build system’s ground truth.
I don’t think this gets you more crash isolation? It’s an “external process” either way.
But this is related to the actual reason why I think asking language server to manage build system leads to worse UX. Broadly, spawning processes from within a language server is scary, because processes can misbehave: your compiler might eat a Zip bomb or your package manager can get stuck in a DNS. And, from within the language server, you don’t really have any way to inform the user that “hey, this thing have been running for 10 minutes and it looks dead”.
In contrast, editors typically already have some sort of UI for managing a list of currently running external processes. So, if it is the editor which manages build system invocation, it can easily draw the “cancel” button right next to the in-progress output to allow the user to kill any runaway builds.
A Build Adapter Protocol that is not under the VSCode UTF16 laden default? :-)
Very informative, I’ve just implemented this in my own config and already I am seeing the benefits. I appreciate you sharing techniques like this.
Zig is really taking over, this is super cool