It’s kind of a side-remark in the post, but if Hoot can do anything the compiler tower can do, that means there is already not only a scheme, but an elisp, an ecmascript, a python, and a Lua. Hopefully this kind of thing renews interest in writing top levels for guile as a VM-of-polyglot
Andy Wingo, who’s the lead of the Hoot project, is also lead developer of Guile. We’re working to make sure that upstream contributions to Guile itself make it into Guile, and whether or not Hoot sits in the Guile repository itself or as a seaparate package we are figuring out as the project progresses.
This is exciting, excellent work! I’d been wondering if compiling Scheme to WASM was even possible—continuations are notoriously hard to implement correctly on any VM not specifically designed for them.
I should try porting Schemepunk to Guile again. Back in 2020 it was blocked by bugs in Guile’s R7RS support, but I think those have since been fixed.
Hoot depends on both the tail call and GC Wasm extensions. On the GC side, Hoot will emit extra instructions to describe its types according to the Wasm GC spec and then the host VM can do the collecting.
I’m not sure where to put it, but the entire subthread is missing a look at the roadmap, which clearly shows that most runtimes do not have TC or GC available yet. Only Chrome can do it, and only with a special flag.
Yes, that’s the current status today. Firefox is in the process of actively implementing both. The proposals themselves appear to be progressing well through the standards process. Two web VMs are required to be implementing a proposal to progress through the later stages. The features should be default enabled once standardised, assuming no obstacles appear. People involved in the proposals process have estimated they should be available by the end of the year.
I believe other non-web engines are also working on these proposals, though I forget which at the moment.
Scheme makes use of tail calls and GC, and those extensions are on track to be generally available in common Wasm engines this year, so it seems like a reasonable design choice to target them now.
IIRC rolling your own GC in WASM is quite awkward/difficult, especially if you consider object references between containers. There was a post about it a month or so ago (i don’t remember the details; maybe it was the previous Spritely post?)
Also, the major WASM runtimes already contain world-class GCs, and run a GC’d language, so exposing those GCs to WASM seems a good idea for performance and interop.
(But I do get your point about piling on features that other WASM runtimes now need to add! Fortunately GC isn’t hard to implement, if you don’t care about world-class performance. I’ve done it twice this year for my smol_world project.)
Ah hmm… For these particular abilities though, it’s at the very least quite hard or potentially impossible to get the same result without some kind of engine support.
For the case of tail calls, it’s much more natural to express recursive programs (esp. ones in Scheme written to expect tail calls) in this style. Perhaps custom stack management at run-time or other hefty program transformations can be used as a workaround, but the concept of tail calls is fairly straightforward and the host engine complexity appears to be small.
For the case of GC, the proposal is far more complex, so I can understand hesitation in terms of complexity… At the same time, allowing Wasm programs to leverage the existing engine GC does make implementation drastically simpler (for languages expecting GC). It importantly also makes it possible to describe cycles between host engines data and Wasm program data which just wasn’t possible before.
I suppose a generalised version of your concern might be that you don’t want every language feature to become a Wasm extension, and I agree with that general sentiment. In the case of tail calls and GC though, they feel (to me at least) sufficiently useful to a variety of languages and allow the Wasm engine to enable a use cases that may be impossible (or very hard) otherwise.
Tail call elimination simplifies things a lot, and there’s very little reason not to have it… how to do it has been known for a long time. GCC even supports it for C in many cases, IIRC.
At any rate, I won’t go into the GC proposal stuff in depth, but here are some good motivators to see that work advance:
It means many more languages being able to become first class citizens in browser-space
It adds certain reference-integrity safety abilities to WASM languages, important and useful for ocap reasons
Usually some kind of efficient GC is already available. In the browser especially. Why not expose it?
It means being able to have a shared heap. This is really useful for garbage collection reasons across programs. Javascript and other languages can instantiate and share references without needing to duplicate or having very difficult to deal with cycle detection and elimination problems.
I’m not super up-to-date, but as I understand it, GC will be in consumer browsers by Q4 2023 (according to Andy Wingo.) And is available in development builds currently, so language implementers may want to start targeting it now.
It’s kind of a side-remark in the post, but if Hoot can do anything the compiler tower can do, that means there is already not only a scheme, but an elisp, an ecmascript, a python, and a Lua. Hopefully this kind of thing renews interest in writing top levels for guile as a VM-of-polyglot
Great stuff! Is it possible to contribute this back into Guile itself? Seems super valuable!
Thanks for your interest and enthusiasm!
Andy Wingo, who’s the lead of the Hoot project, is also lead developer of Guile. We’re working to make sure that upstream contributions to Guile itself make it into Guile, and whether or not Hoot sits in the Guile repository itself or as a seaparate package we are figuring out as the project progresses.
This is exciting, excellent work! I’d been wondering if compiling Scheme to WASM was even possible—continuations are notoriously hard to implement correctly on any VM not specifically designed for them.
I should try porting Schemepunk to Guile again. Back in 2020 it was blocked by bugs in Guile’s R7RS support, but I think those have since been fixed.
Cool stuff! I take it the WASM code uses tail-call optimization?
From earlier posts I’ve seen about Lisp on WASM, it sounds like GC will be a significant hurdle. Hoot will have to implement that itself, right?
Hoot depends on both the tail call and GC Wasm extensions. On the GC side, Hoot will emit extra instructions to describe its types according to the Wasm GC spec and then the host VM can do the collecting.
I’m not sure where to put it, but the entire subthread is missing a look at the roadmap, which clearly shows that most runtimes do not have TC or GC available yet. Only Chrome can do it, and only with a special flag.
Yes, that’s the current status today. Firefox is in the process of actively implementing both. The proposals themselves appear to be progressing well through the standards process. Two web VMs are required to be implementing a proposal to progress through the later stages. The features should be default enabled once standardised, assuming no obstacles appear. People involved in the proposals process have estimated they should be available by the end of the year.
I believe other non-web engines are also working on these proposals, though I forget which at the moment.
Hmm, that’s unfortunate
Unfortunate in what way…?
Scheme makes use of tail calls and GC, and those extensions are on track to be generally available in common Wasm engines this year, so it seems like a reasonable design choice to target them now.
Ideally those things would be handled internally, though, not rely on extra features being added to every WASM engine.
IIRC rolling your own GC in WASM is quite awkward/difficult, especially if you consider object references between containers. There was a post about it a month or so ago (i don’t remember the details; maybe it was the previous Spritely post?)
Also, the major WASM runtimes already contain world-class GCs, and run a GC’d language, so exposing those GCs to WASM seems a good idea for performance and interop.
(But I do get your point about piling on features that other WASM runtimes now need to add! Fortunately GC isn’t hard to implement, if you don’t care about world-class performance. I’ve done it twice this year for my smol_world project.)
Ah hmm… For these particular abilities though, it’s at the very least quite hard or potentially impossible to get the same result without some kind of engine support.
For the case of tail calls, it’s much more natural to express recursive programs (esp. ones in Scheme written to expect tail calls) in this style. Perhaps custom stack management at run-time or other hefty program transformations can be used as a workaround, but the concept of tail calls is fairly straightforward and the host engine complexity appears to be small.
For the case of GC, the proposal is far more complex, so I can understand hesitation in terms of complexity… At the same time, allowing Wasm programs to leverage the existing engine GC does make implementation drastically simpler (for languages expecting GC). It importantly also makes it possible to describe cycles between host engines data and Wasm program data which just wasn’t possible before.
I suppose a generalised version of your concern might be that you don’t want every language feature to become a Wasm extension, and I agree with that general sentiment. In the case of tail calls and GC though, they feel (to me at least) sufficiently useful to a variety of languages and allow the Wasm engine to enable a use cases that may be impossible (or very hard) otherwise.
Tail call elimination simplifies things a lot, and there’s very little reason not to have it… how to do it has been known for a long time. GCC even supports it for C in many cases, IIRC.
At any rate, I won’t go into the GC proposal stuff in depth, but here are some good motivators to see that work advance:
Oh, so WASM GC is available already? I had the impression it was a ways out.
Wasm GC is experimentally available via flags or being implemented in at least Chrome and Firefox, perhaps some non-browser implementations as well.
It’s believed to be on track to be generally available in stable browsers and engines sometime this year IIRC.
I’m not super up-to-date, but as I understand it, GC will be in consumer browsers by Q4 2023 (according to Andy Wingo.) And is available in development builds currently, so language implementers may want to start targeting it now.