I’ve been under the impression that Racket-on-Chez is mostly a refactoring project to reduce the amount of C code. There are some low-level stuff that you have to do in C, but they want to do as much intermediate-level stuff as possible in a high-level Lisp language.
Chez Scheme happens to provide a good foundation for all the low-level stuff and some intermediate-level stuff, so they chose it as a basis.
Right but the point of the article is that they have to maintain Chez scheme fork now too? They don’t get that for free, but rather “inherit” it. That could be better than not doing it, of course.
I looked at the Racket source code when this project started, and I was surprised to learn that it was indeed hundreds of thousands of lines of C code, just like Python, Ruby, Perl, and R! For some reason I had assumed that Lisp dialects would be more bootstrapped :)
I guess you can’t underestimate the speed of C, and if you want to get rid of that, you have to replace it with something pretty advanced like Chez Scheme, which has its own way of solving the portability problem. AFAIU that’s why Cisco uses it and acquired it… because the Chez Scheme compiler supports a lot of weird hardware variants.
Right but the point of the article is that they have to maintain Chez scheme fork now too?
Yes, but this article seems to suggest that this is not much better than before. In fact it is much better than before:
There is much less C code
A lot of intermediate-level stuff is now in Scheme
Even though it’s a fork, it still largely resembles the Chez Scheme codebase, so anyone who has worked in Chez Scheme’s codebase will have an easier time contributing to Racket’s codebase
Compare this the BSD situation: the 3 major BSDs (FreeBSD, OpenBSD, FreeBSD) forked a long time ago, and they never intend to keep full compatibility with each other; but their code is still similar enough for developers to cherrypick changes, and for contributors to move from one BSD to another.
Wow it’s crazy that Chez Scheme is from 1984 and Racket is from 1995!
I can definitely see that maintaining a big language implementation is a huge thing to ask, especially for 5 CS professors who are doing many other things like teaching, reviewing papers, serving on committees, etc.
Even better, Chez Scheme goes back to Z80 processors. Many folks said you must stick entirely with C on weak hardware. They schemed up something better. Their historical paper (pdf) was full of interesting stuff.
Below is a copy of a comment I made on Reddit re. the post.
Some patches that Racket needed were detrimental for Chez Scheme at large.
I’m curious which patches MB is referring to here. “Detrimental” to me implies making CS worse, but I’d be surprised if there were any such patches. There are definitely patches containing changes that the CS folks aren’t interested in, but that doesn’t mean they’re not good changes. That seems to me more of a difference in values (i.e. a subjective thing) rather than a judgement on whether the changes are good or bad. It can also be a matter of the effort required by the CS team to understand and merge some of the more complicated patches.
But the actual project undertaken by the Racket team was to preserve the existing behavior of Racket and all its libraries.
[…]
And instead of capitulating to the Chez Scheme way—say, by removing or changing certain features of Racket—these Racket-compatibility patches became the basis of the Racket-specific fork of Chez Scheme.
I’ve been a Racket user for far less than MB, but to me the opposite would’ve been a lot more surprising. Of course a language that values backwards-compatibility is going to do its best not to break it.
Single flonums were removed because nobody was using them so Racket CS did break backwards-compatibility in areas where the costs outweighed the benefits. I think there were a couple more small examples like this, but I can’t remember them off the top of my head. In areas where changes were needed to preserve the same level of performance as Racket BC (eg. unboxed flonums), those changes also benefited CS. Parallel and incremental GC is another one like that, but that’s one that hasn’t yet made it into CS.
I thought the broader objective was to permanently combine the greatest assets of Chez Scheme (= its speed and stability) with the greatest assets of Racket (= its language-oriented programming facilities, including its macro system and module system). This didn’t happen.
I think this did happen. Racket CS is generally faster than Racket BC, it is more maintainable by more people and it’s still Racket.
As of today, the Racket fork of Chez Scheme is “1249 commits ahead, 617 commits behind” the main branch of Chez Scheme
1249 is very close to the total number of commits in the racket fork; and 617 is very close to the total number of commits in the chez scheme repository. So I think this is indicative of a history rewrite and doesn’t actually measure the difference between the two codebases.
I thought the goal was to move Racket onto Chez Scheme and accept whatever losses to backward compatibility might arise, on the idea that the future benefits were so terrific. This is a tradeoff that happens all the time in programming.
The fork of Chez Scheme is claimed to have achieved 2 goals: not breaking backward compatibility with Racket, and hacking the performance so that the new implementation is as fast or faster than the previous implementation. I don’t know about the performance code changes, but it’s possible that some of these were tradeoffs that favoured Racket use cases at the cost of disadvantaging Cisco use cases. If avoiding a fork would have caused a loss of backward compatibility and weird performance regressions, then likely this would have caused a schism similar to Python 2 / Python 3, and the core team is probably wise to have avoided this.
If the new version is even more maintainable by avoiding a fork, but you cause a schism, which requires the old version to be maintained in parallel with the new version, then this is a net loss in maintainability.
I’ve been under the impression that Racket-on-Chez is mostly a refactoring project to reduce the amount of C code. There are some low-level stuff that you have to do in C, but they want to do as much intermediate-level stuff as possible in a high-level Lisp language.
Chez Scheme happens to provide a good foundation for all the low-level stuff and some intermediate-level stuff, so they chose it as a basis.
If you look at the comparison diagram on page 78:3 of https://www.cs.utah.edu/plt/publications/icfp19-fddkmstz.pdf, they have clearly achieved that goal.
Right but the point of the article is that they have to maintain Chez scheme fork now too? They don’t get that for free, but rather “inherit” it. That could be better than not doing it, of course.
I looked at the Racket source code when this project started, and I was surprised to learn that it was indeed hundreds of thousands of lines of C code, just like Python, Ruby, Perl, and R! For some reason I had assumed that Lisp dialects would be more bootstrapped :)
I guess you can’t underestimate the speed of C, and if you want to get rid of that, you have to replace it with something pretty advanced like Chez Scheme, which has its own way of solving the portability problem. AFAIU that’s why Cisco uses it and acquired it… because the Chez Scheme compiler supports a lot of weird hardware variants.
Yes, but this article seems to suggest that this is not much better than before. In fact it is much better than before:
Compare this the BSD situation: the 3 major BSDs (FreeBSD, OpenBSD, FreeBSD) forked a long time ago, and they never intend to keep full compatibility with each other; but their code is still similar enough for developers to cherrypick changes, and for contributors to move from one BSD to another.
SBCL is self hosted iirc
Wow it’s crazy that Chez Scheme is from 1984 and Racket is from 1995!
I can definitely see that maintaining a big language implementation is a huge thing to ask, especially for 5 CS professors who are doing many other things like teaching, reviewing papers, serving on committees, etc.
Even better, Chez Scheme goes back to Z80 processors. Many folks said you must stick entirely with C on weak hardware. They schemed up something better. Their historical paper (pdf) was full of interesting stuff.
Below is a copy of a comment I made on Reddit re. the post.
I’m curious which patches MB is referring to here. “Detrimental” to me implies making CS worse, but I’d be surprised if there were any such patches. There are definitely patches containing changes that the CS folks aren’t interested in, but that doesn’t mean they’re not good changes. That seems to me more of a difference in values (i.e. a subjective thing) rather than a judgement on whether the changes are good or bad. It can also be a matter of the effort required by the CS team to understand and merge some of the more complicated patches.
I’ve been a Racket user for far less than MB, but to me the opposite would’ve been a lot more surprising. Of course a language that values backwards-compatibility is going to do its best not to break it.
Single flonums were removed because nobody was using them so Racket CS did break backwards-compatibility in areas where the costs outweighed the benefits. I think there were a couple more small examples like this, but I can’t remember them off the top of my head. In areas where changes were needed to preserve the same level of performance as Racket BC (eg. unboxed flonums), those changes also benefited CS. Parallel and incremental GC is another one like that, but that’s one that hasn’t yet made it into CS.
I think this did happen. Racket CS is generally faster than Racket BC, it is more maintainable by more people and it’s still Racket.
1249 is very close to the total number of commits in the racket fork; and 617 is very close to the total number of commits in the chez scheme repository. So I think this is indicative of a history rewrite and doesn’t actually measure the difference between the two codebases.
Unfortunately, it probably is a useful indicator of the likelihood that racket could ever be based on mainline Chez, isn’t it?
Not necessarily, history rewrites could be done for whatever silly reason. You need to diff the actual trees.
The original Chez repo keeps binary boot files in the repo, and the Racket fork presumably rewrote history to remove those boot files.
If you try to clone both versions you’ll notice that Racket’s version is much smaller (40MB vs 1.2GB).
The fork of Chez Scheme is claimed to have achieved 2 goals: not breaking backward compatibility with Racket, and hacking the performance so that the new implementation is as fast or faster than the previous implementation. I don’t know about the performance code changes, but it’s possible that some of these were tradeoffs that favoured Racket use cases at the cost of disadvantaging Cisco use cases. If avoiding a fork would have caused a loss of backward compatibility and weird performance regressions, then likely this would have caused a schism similar to Python 2 / Python 3, and the core team is probably wise to have avoided this.
If the new version is even more maintainable by avoiding a fork, but you cause a schism, which requires the old version to be maintained in parallel with the new version, then this is a net loss in maintainability.