I understand that POV, but wouldn’t necessarily say that with regards to this project - glue languages are pretty common, so having one for async rust is something you can come up with naturally without any specific program in mind
I was expecting some explanation of how async would be exposed in scheme. The perhaps that is for a future post. In Piccolo async is used under the hood for stackless coroutines in Lua. I thought that was a really cool application.
It’s pretty simple. Calling an async bridge function automatically awaits it. However, we also allow for Scheme values to be Futures, which are shared, and those must be awaited via an await builtin.
Oh wow, that sounds very cool! I have recently fallen back on my parenthesis addiction and looked into Scheme languages again. I was actually wondering if/when there will be a scheme written in Rust (even though there probably already exist several, but I never bothered to actually look).
Is there a reason to prefer R6RS over R7RS? I know that not everyone is happy with the newer standard, but I was wondering if you have an opinion on that matter.
Thanks! There are a few other Scheme implementations written in Rust I think, but they are less complete and not actively worked on I believe (and of course they don’t support interoperability with async rust)
The reason R6RS is chosen is because it’s much more complete of a spec, as of right now R7RS small is the only version of R7RS released, and that spec does not include some desirable things like syntax-case. When it is fully released, the intention is to support R7RS large
Unrelated to Rust scheme implementations, but when you stated that the primary reason was async, I was reminded of lispx, which I think is a pretty cool project. It was also created to solve the ergonomics of async in its host language. Maybe there’s something there that could be of inspiration. I enjoy a scheme as much as the next parenthesis-loving person, but I’d be happy to see more kernel implementations.
By way of brief comparison, besides the async compatability, Scheme-rs has a much stronger macro system than both of these impls at the current moment, which are attempting to be compatible with R7RS small or R4RS. For example, steel cannot define a macro of the following form:
(define-syntax loop
(lambda (x)
(syntax-case x ()
[(k e ...)
(with-syntax
([break (datum->syntax #'k 'break)])
#'(call-with-current-continuation
(lambda (break)
(let f () e ... (f)))))])))
to be fair to these implementations, they have a lot of features that scheme-rs lacks. But all of the ones present in R6RS I intend to add sooner rather than later
This is very exciting to see - native compilation is something I’ve been wanting to work on for steel for a long time, I’m happy to see someone else take a stab at it for their scheme. I’ve been toying around with cranelift for years but haven’t had the dedicated time to push through the work on it
To address two of the points on syntax-case and async:
As of last night, steel can define this:
(define-syntax (loop x)
(syntax-case x ()
[(k e ...)
(with-syntax ([break #'k])
#'(call-with-current-continuation (lambda (break)
(let f ()
e
...
(f)))))]))
Just have to work on the datum->syntax and then there is parity. The syntax-case implementation was slightly broken in the mainline for a while (and also, undocumented).
For async, steel does have support for rust async functions registered against the runtime, and I have a fun way of integrating with an external executor instance via continuations, but it does not have an async public api (yet) like you do. Best of luck with this and I’m excited to see where it goes
Oh that’s sweet! Very cool to see syntax-case in steel!
I hope I wasn’t missrepresenting your project! Let me be clear here: Steel is faster and more featureful than Scheme-rs by a long shot. I brought up syntax case because that’s were the initial bulk of my effort went (the remainder went to reimplementing the project as a compiled)
It absolutely is the hardest thing to properly implement in my opinion (and I’m technically not even fully compliant, I don’t add any continuations to the expander (soooo much work)).
I’m thankful I spent so much time considering it while the eval logic was interpreted, it was a weight off my shoulder when moving to compiling. That being said, compiling it also lead to a new set of challenges, I had to somehow figure out a way to capture the environment, something that I was just throwing out after compilation.
As an aside, have you noticed how there are so many people who work with Scheme named “Matt”? It’s starting to freak me out a bit
I mean you can get a lot of the Rust vibe without the parentheses from Gleam and supposedly that should have a more than decent async story (though I haven’t looked into the erlang runtime yet).
Would scheme-rs be a good choice for adding general purpose runtime scripting to Rust applications (in sync or async Rust)?
Would scheme-rs be suitable for running untrusted user provided scripts? This entails limiting what a script has access to, and limiting runtime and memory usage.
Would you say scheme-rs’s code is mature enough that 3rd parties could jump in and start contributing? Or is the project still in enough flux that it would be difficult?
Yes, specifically for async right now. Although I do want to provide a sync interface at some point.
To some extent. Limiting what the script has access to, absolutely. You get to decide what functions an environment can access. However there are no mechanisms for limiting memory usage or runtime at the moment.
Yes. Most of the architecture is fixed at this point. Some things will change, i.e. the value enum will eventually become opaque so that we can properly optimize it, but since I wrote this post a couple of people have begun to contribute to the project with no issues
Haha I see now what you meant! And indeed, that was the original reason :-) language implementation has always been my passion so I wanted to see how far I could get into implementing a language to the specification. I picked Scheme R6RS because I thought it would be easy and small (how wrong I was!)
Further evidence for my hypothesis that the natural end point of every program design is a language.
I understand that POV, but wouldn’t necessarily say that with regards to this project - glue languages are pretty common, so having one for async rust is something you can come up with naturally without any specific program in mind
Is that pro-DSL propaganda or something else?
I have a brilliant proof of this that I can’t possibly fit into the margin.
Obviously because making languages is simply the most fun part of programming.
Or alternatively another example of Greenspun’s tenth rule ( but generalised to be any lisp family language not just CL):
Perhaps we can just amend that to say C, Fortran or Rust.
Hey! Scheme-rs is formally-specified!
I was expecting some explanation of how async would be exposed in scheme. The perhaps that is for a future post. In Piccolo async is used under the hood for stackless coroutines in Lua. I thought that was a really cool application.
It’s pretty simple. Calling an async bridge function automatically awaits it. However, we also allow for Scheme values to be Futures, which are shared, and those must be awaited via an
awaitbuiltin.I haven’t converted these back over yet (so much WIP stuff going on), but you can see how the spawn, await, sleep, and join bridge functions are defined here: https://github.com/maplant/scheme-rs/blob/main/src/futures.rs
Yes. Yes. Yes. I have been wanting this for so long as a scripting language in Rust apps for so long.
I hope this can be useful for you! Let me know if there are any particular shortcomings for your use case and I will expedite fixing them :-)
Oh wow, that sounds very cool! I have recently fallen back on my parenthesis addiction and looked into Scheme languages again. I was actually wondering if/when there will be a scheme written in Rust (even though there probably already exist several, but I never bothered to actually look).
Is there a reason to prefer R6RS over R7RS? I know that not everyone is happy with the newer standard, but I was wondering if you have an opinion on that matter.
Thanks! There are a few other Scheme implementations written in Rust I think, but they are less complete and not actively worked on I believe (and of course they don’t support interoperability with async rust)
The reason R6RS is chosen is because it’s much more complete of a spec, as of right now R7RS small is the only version of R7RS released, and that spec does not include some desirable things like syntax-case. When it is fully released, the intention is to support R7RS large
There are at least two (three now) being actively developed that I know about.
Unrelated to Rust scheme implementations, but when you stated that the primary reason was async, I was reminded of lispx, which I think is a pretty cool project. It was also created to solve the ergonomics of async in its host language. Maybe there’s something there that could be of inspiration. I enjoy a scheme as much as the next parenthesis-loving person, but I’d be happy to see more kernel implementations.
Ah I see! I think I was thinking of https://github.com/volution/vonuvoli-scheme which last saw an update two years ago.
By way of brief comparison, besides the async compatability, Scheme-rs has a much stronger macro system than both of these impls at the current moment, which are attempting to be compatible with R7RS small or R4RS. For example, steel cannot define a macro of the following form:
to be fair to these implementations, they have a lot of features that scheme-rs lacks. But all of the ones present in R6RS I intend to add sooner rather than later
Disclaimer: steel is my project
This is very exciting to see - native compilation is something I’ve been wanting to work on for steel for a long time, I’m happy to see someone else take a stab at it for their scheme. I’ve been toying around with cranelift for years but haven’t had the dedicated time to push through the work on it
To address two of the points on syntax-case and async:
As of last night, steel can define this:
Just have to work on the
datum->syntaxand then there is parity. The syntax-case implementation was slightly broken in the mainline for a while (and also, undocumented).For async, steel does have support for rust async functions registered against the runtime, and I have a fun way of integrating with an external executor instance via continuations, but it does not have an async public api (yet) like you do. Best of luck with this and I’m excited to see where it goes
Oh that’s sweet! Very cool to see syntax-case in steel!
I hope I wasn’t missrepresenting your project! Let me be clear here: Steel is faster and more featureful than Scheme-rs by a long shot. I brought up syntax case because that’s were the initial bulk of my effort went (the remainder went to reimplementing the project as a compiled)
All good - the devil is in the (lack of) documentation :)
Syntax-case was a thorn in my side for years, I only recently put in the effort to get it functioning, I found it a bit mind bendy to get right
It absolutely is the hardest thing to properly implement in my opinion (and I’m technically not even fully compliant, I don’t add any continuations to the expander (soooo much work)).
I’m thankful I spent so much time considering it while the eval logic was interpreted, it was a weight off my shoulder when moving to compiling. That being said, compiling it also lead to a new set of challenges, I had to somehow figure out a way to capture the environment, something that I was just throwing out after compilation.
As an aside, have you noticed how there are so many people who work with Scheme named “Matt”? It’s starting to freak me out a bit
I mean you can get a lot of the Rust vibe without the parentheses from Gleam and supposedly that should have a more than decent async story (though I haven’t looked into the erlang runtime yet).
Why would I want that? Jokes aside Gleam is a fantastic language but not what I want. I want an embedded glue language for async rust.
I have a few questions:
Would scheme-rs be a good choice for adding general purpose runtime scripting to Rust applications (in sync or async Rust)?
Would scheme-rs be suitable for running untrusted user provided scripts? This entails limiting what a script has access to, and limiting runtime and memory usage.
Would you say scheme-rs’s code is mature enough that 3rd parties could jump in and start contributing? Or is the project still in enough flux that it would be difficult?
Yes, specifically for async right now. Although I do want to provide a sync interface at some point.
To some extent. Limiting what the script has access to, absolutely. You get to decide what functions an environment can access. However there are no mechanisms for limiting memory usage or runtime at the moment.
Yes. Most of the architecture is fixed at this point. Some things will change, i.e. the value enum will eventually become opaque so that we can properly optimize it, but since I wrote this post a couple of people have begun to contribute to the project with no issues
Great, I will try looking into adding a resource limiting feature :)
Very cool! This should certainly be possible, although obviously a bit of an effort.
To answer the title question: Why should you not?
I’m not sure what your comment is intended to say. Can you elaborate please?
Sure! It is a positive comment. You want to implement Scheme just because you want to implement it! There’s no “why”, like for example:
You never ask someone “Why did you paint a rose?”. You congratulate them and ask for a bouquet next
Haha I see now what you meant! And indeed, that was the original reason :-) language implementation has always been my passion so I wanted to see how far I could get into implementing a language to the specification. I picked Scheme R6RS because I thought it would be easy and small (how wrong I was!)