Pretty amazing and great work about something people often overlook — how do we ensure we have something clean in our toolchains.
I’m also wondering how the story will look in 30 years from now — would there be a need to compile GCC suite at all, or majority of software will be written in different set of languages, like Go, Rust, …
Even today, you might get away with using some other compiler in place of GCC. Like, clang or llvm or something. Uh, I don’t know the details here; not my area of expertise!
But, that doesn’t change the nature of the situation you’re describing–it only changes the particulars. Substituting one big compiler project for another doesn’t scratch that itch.
This script installs from sources latest released version of Go without any prior Go compiler available in roughly 20 minutes (most of that time is spent in selftests of Go) on my weak machine.
Awesome achievement. Many modern languages become self-hosting as they mature, and this means they lose bootstrappability, because maintaining two implementations is not fun. Understandable, but sad, and I hope guix makes inroads there too.
I don’t think becoming self-hosting automatically makes a language unbootstrappable, because you could always compile the pre-self-hosted version to bootstrap the self-hosted versions. Though I think this could be broken by a major shift in hardware, where a future system trying to bootstrap itself in new hardware has no way to run old C compilers, so you need to use the self-hosted compiler of a language to cross-compile the compiler to your new hardware and use the binary for bootstrapping from that day forward.
I really, really wish that it were being done in Lisp rather than Scheme, but that’s an effect of rms’s years-long lack of appreciation for Common Lisp. It’s a crying shame.
Why? Scheme, specifically Guile has a much nicer interface for working with the operating system, since it isn’t as constrained by portability-standardisation as CL.
I’d really have to write a massive article about why Scheme is IMHI unsuitable for building large systems.
In part it’s the small standard. By the time an implementation adds enough features to be capable of implementing a large system any such system would be importable. Lisp has packages and CLOS, and that alone counts for a lot, but even stuff like LOOP and FORMAT are important.
In part it’s features like call/cc. Brilliant, really smart, amazingly powerful — but it makes certain efficient operations either altogether impossible or a pain to write. And dynamic-wind is broken.
Then there’s stuff like having a single namespace. Makes a lot of sense, very similar to other languages — but ultimately a mistake. A key insight of Lisp is that programmers can add namespaces to the language, and it can all work just fine.
Then there are matters of taste like adding #t and #f to the language, making nil non-false and making (car nil) an error. Those last two, in particular, are simply inexcusable IMHO.
That’s not to ignore the misfeatures of Lisp. Pathnames are woefully under-specified and default up-casing is in almost as poor taste as Scheme’s nil abuse. Almost.
In short, I just cringe any time I see someone use Scheme for real work. For teaching or playing around it’s almost indispensable. It has some great ideas, most of which made it into Common Lisp. But it is not a suitable language for real systems programming. IMHO anyway😀
Having used Scheme quite a bit and not really touched CL, I can’t really imagine #f and nil being smooshed into one. They’re distinct values of distinct types with distinct uses. And what does (car nil) give, and what do you gain over an erroring implementation?
I don’t think as bold a claim as “this language is unsuitable for real systems programming” can be guarded with an “IMHO” - if it was just an opinion, you would be more aware of the matter of taste that the choice between these rather similar languages is, and not phrasing it as if you know that this particular language is unsuitable for large systems while commenting on a counter-example by your own admission.
Having used Scheme quite a bit and not really touched CL, I can’t really imagine #f and nil being smooshed into one. They’re distinct values of distinct types with distinct uses.
Eh, it works out really well. NIL is always false, so comparisons always return it or a true value. Because the uses are distinct, it doesn’t cause the problems you might imagine.
And what does (car nil) give, and what do you gain over an erroring implementation?
It returns NIL, which is nice because you can often write more elegant code. I tried to Google an example for you, but my go-to source is down, sorry.
I don’t think that Guix or any large Scheme system is a counter-example. After all, people write large systems in C and C++! It’s not that it’s impossible, just that it’s suboptimal.
In part it’s the small standard. By the time an implementation adds enough features to be capable of implementing a large system any such system would be importable. Lisp has packages and CLOS, and that alone counts for a lot, but even stuff like LOOP and FORMAT are important.
Then again, Guix specifically isn’t meant to be portable across different Schemes, but (afaik) limits itself to Guile. So you do have stuff like GOOPS and format, though not loop (as of now).
All your other points seem to be ranking Scheme as a Lisp, which it really just isn’t. I guess that’s the problem, if you insist on seeing it that way?
All your other points seem to be ranking Scheme as a Lisp, which it really just isn’t. I guess that’s the problem, if you insist on seeing it that way?
Exactly. Lisp is a great programming language in which to implement large, interlocking systems, while Scheme is a great programming language for reasoning about computation. Since Guix is a large system, or perhaps even a large interlocking system of systems, it would IMHO be better implemented in Lisp than in Scheme.
Hence my original post😀
I really do believe the world would be better with a Common Lisp GNU Emacs, a Common Lisp Firefox, a Common Lisp st, StumpWM, a Common Lisp X.org or Wayland and a Common Lisp Linux. Maybe instead of isolated processes we would have isolated global environments, such as Robert Strandh has worked on.
Anyway, reading about the ecosystem that Guix is putting together — which really is neat and cool and impressive and a great step towards a partially-Lisp ecosystem — really made me wish that rms didn’t have such a dislike for Common Lisp. If only Emacs had been rewritten in Common Lisp two decades ago!
I never worked on either a large CL or Scheme/Guile project, so I really cannot tell. Until now I haven’t seen why one would be more or less fit than the other.
But that’s not that important. I was wondering if you could give me some pointers on what you mention
Maybe instead of isolated processes we would have isolated global environments, such as Robert Strandh has worked on.
He proposes a protocol for multiple global environments within a single Lisp image; since lookups terminate in the global environment, this means that unless one environment is deliberately exposed to another (say, by a higher-privileged environment) then each environment is isolated from all others.
It’s related to how one can use a cons cell in Lisp as a non-algebraic pointer. If you imagine a Lisp with no pointers to core memory (where only the implementation itself can access raw RAM), then you could imagine an initial environment which sets up smaller sandboxed subenvironments, each of which is kinda, if you squint right, a bit like a process, able to message other environments only in the ways permitted by the initial environment.
I really can’t wait until I have the free time to work on this!
interface for working with the operating system, since it isn’t as constrained by portability-standardisation as CL.
Why are you under the impression that the standard constrains the APIs offered by Lisp implementations? SBCL comes with the sb-posix package which provides a nice interface to the OS APIs like sockets, file attributes (stat) etc.
I don’t know about the APIs offered by guile so I can’t compare them though.
Hm, I haven’t used sb-posix, but the reason I’m under that impression is first of all that I was working on a StumpWM extension earlier this year, and so I couldn’t limit myself to any particular implementation. I was re-writing the maildir package, so there was a lot of looking up directories and so on, and it really didn’t seem intuitive at first. Maybe it was just a bad example, or I didn’t read into it as much.
But this is the Guile package for POSIX, built in by default. I haven’t had the opportunity to properly use all of it yet, but to me it seems very clean.
I was working on a StumpWM extension earlier this year, and so I couldn’t limit myself to any particular implementation.
This is tangential to the OP but I help maintain StumpWM, I think I remember your extension. Was yours the one to support multiple maildirs? Anyway you most certainly can limit yourself to a SBCL when working for StumpWM. We dropped support for anything but SBCL a couple of years ago. One of the reasons I was in favor of only supporting SBCL was precisely because we could use stuff in sb-posix to interact with the OS more easily.
there was a lot of looking up directories and so on, and it really didn’t seem intuitive at first. Maybe it was just a bad example
I think it is a good example, precisely because the API in the CL standard for manipulating directories sucks, or at least it sucks if you are in UNIX. From reading the documentation to the POSIX API of guile it is similar to the one offered in sb-posix. But it shows what I was getting at: The standard only provides a ‘lower ceiling’ for the APIs offered by different implementations.
Anyway you most certainly can limit yourself to a SBCL when working for StumpWM. We dropped support for anything but SBCL a couple of years ago.
Oh, I thought that I had seen portable code in another extension, making me think that all code should be portable? Either way, even if it is possible, I always consider it a pity to use such extensions, even if it’s pragmatic.
Yes, but I think he was a major force in keeping Emacs Lisp as different from Common Lisp as it is, as well as in deciding on Scheme for the official GNU extension language.
Pretty amazing and great work about something people often overlook — how do we ensure we have something clean in our toolchains. I’m also wondering how the story will look in 30 years from now — would there be a need to compile GCC suite at all, or majority of software will be written in different set of languages, like Go, Rust, …
Even today, you might get away with using some other compiler in place of GCC. Like, clang or llvm or something. Uh, I don’t know the details here; not my area of expertise!
But, that doesn’t change the nature of the situation you’re describing–it only changes the particulars. Substituting one big compiler project for another doesn’t scratch that itch.
The Rust compiler is currently written in Rust, so it has a bootstrapping problem, just like GCC does. See this fine article about that: https://guix.gnu.org/blog/2018/bootstrapping-rust/
I don’t know about golang. Hm,
apt-get build-dep golang-gotries to pull in GCC. So, that’s a data point.The current golang is built using go. But a previous version (1.4?) was built using C. So that is used for bootstrapping.
This script installs from sources latest released version of Go without any prior Go compiler available in roughly 20 minutes (most of that time is spent in selftests of Go) on my weak machine.
Awesome achievement. Many modern languages become self-hosting as they mature, and this means they lose bootstrappability, because maintaining two implementations is not fun. Understandable, but sad, and I hope guix makes inroads there too.
I don’t think becoming self-hosting automatically makes a language unbootstrappable, because you could always compile the pre-self-hosted version to bootstrap the self-hosted versions. Though I think this could be broken by a major shift in hardware, where a future system trying to bootstrap itself in new hardware has no way to run old C compilers, so you need to use the self-hosted compiler of a language to cross-compile the compiler to your new hardware and use the binary for bootstrapping from that day forward.
That’s so cool and such amazing work!
I really, really wish that it were being done in Lisp rather than Scheme, but that’s an effect of rms’s years-long lack of appreciation for Common Lisp. It’s a crying shame.
Why? Scheme, specifically Guile has a much nicer interface for working with the operating system, since it isn’t as constrained by portability-standardisation as CL.
I’d really have to write a massive article about why Scheme is IMHI unsuitable for building large systems.
In part it’s the small standard. By the time an implementation adds enough features to be capable of implementing a large system any such system would be importable. Lisp has packages and CLOS, and that alone counts for a lot, but even stuff like
LOOPandFORMATare important.In part it’s features like
call/cc. Brilliant, really smart, amazingly powerful — but it makes certain efficient operations either altogether impossible or a pain to write. Anddynamic-windis broken.Then there’s stuff like having a single namespace. Makes a lot of sense, very similar to other languages — but ultimately a mistake. A key insight of Lisp is that programmers can add namespaces to the language, and it can all work just fine.
Then there are matters of taste like adding
#tand#fto the language, makingnilnon-false and making(car nil)an error. Those last two, in particular, are simply inexcusable IMHO.That’s not to ignore the misfeatures of Lisp. Pathnames are woefully under-specified and default up-casing is in almost as poor taste as Scheme’s
nilabuse. Almost.In short, I just cringe any time I see someone use Scheme for real work. For teaching or playing around it’s almost indispensable. It has some great ideas, most of which made it into Common Lisp. But it is not a suitable language for real systems programming. IMHO anyway😀
Having used Scheme quite a bit and not really touched CL, I can’t really imagine
#fandnilbeing smooshed into one. They’re distinct values of distinct types with distinct uses. And what does(car nil)give, and what do you gain over an erroring implementation?I don’t think as bold a claim as “this language is unsuitable for real systems programming” can be guarded with an “IMHO” - if it was just an opinion, you would be more aware of the matter of taste that the choice between these rather similar languages is, and not phrasing it as if you know that this particular language is unsuitable for large systems while commenting on a counter-example by your own admission.
Eh, it works out really well.
NILis always false, so comparisons always return it or a true value. Because the uses are distinct, it doesn’t cause the problems you might imagine.It returns
NIL, which is nice because you can often write more elegant code. I tried to Google an example for you, but my go-to source is down, sorry.I don’t think that Guix or any large Scheme system is a counter-example. After all, people write large systems in C and C++! It’s not that it’s impossible, just that it’s suboptimal.
But hey, Scheme is better than C😀
Then again, Guix specifically isn’t meant to be portable across different Schemes, but (afaik) limits itself to Guile. So you do have stuff like GOOPS and
format, though not loop (as of now).All your other points seem to be ranking Scheme as a Lisp, which it really just isn’t. I guess that’s the problem, if you insist on seeing it that way?
Exactly. Lisp is a great programming language in which to implement large, interlocking systems, while Scheme is a great programming language for reasoning about computation. Since Guix is a large system, or perhaps even a large interlocking system of systems, it would IMHO be better implemented in Lisp than in Scheme.
Hence my original post😀
I really do believe the world would be better with a Common Lisp GNU Emacs, a Common Lisp Firefox, a Common Lisp
st, StumpWM, a Common Lisp X.org or Wayland and a Common Lisp Linux. Maybe instead of isolated processes we would have isolated global environments, such as Robert Strandh has worked on.Anyway, reading about the ecosystem that Guix is putting together — which really is neat and cool and impressive and a great step towards a partially-Lisp ecosystem — really made me wish that rms didn’t have such a dislike for Common Lisp. If only Emacs had been rewritten in Common Lisp two decades ago!
I never worked on either a large CL or Scheme/Guile project, so I really cannot tell. Until now I haven’t seen why one would be more or less fit than the other.
But that’s not that important. I was wondering if you could give me some pointers on what you mention
here?
His paper is page 70-something here: https://www.european-lisp-symposium.org/static/proceedings/2015.pdf
He proposes a protocol for multiple global environments within a single Lisp image; since lookups terminate in the global environment, this means that unless one environment is deliberately exposed to another (say, by a higher-privileged environment) then each environment is isolated from all others.
It’s related to how one can use a cons cell in Lisp as a non-algebraic pointer. If you imagine a Lisp with no pointers to core memory (where only the implementation itself can access raw RAM), then you could imagine an initial environment which sets up smaller sandboxed subenvironments, each of which is kinda, if you squint right, a bit like a process, able to message other environments only in the ways permitted by the initial environment.
I really can’t wait until I have the free time to work on this!
Thanks, sounds very interesting!
Why are you under the impression that the standard constrains the APIs offered by Lisp implementations? SBCL comes with the sb-posix package which provides a nice interface to the OS APIs like sockets, file attributes (stat) etc.
I don’t know about the APIs offered by guile so I can’t compare them though.
Hm, I haven’t used sb-posix, but the reason I’m under that impression is first of all that I was working on a StumpWM extension earlier this year, and so I couldn’t limit myself to any particular implementation. I was re-writing the maildir package, so there was a lot of looking up directories and so on, and it really didn’t seem intuitive at first. Maybe it was just a bad example, or I didn’t read into it as much.
But this is the Guile package for POSIX, built in by default. I haven’t had the opportunity to properly use all of it yet, but to me it seems very clean.
This is tangential to the OP but I help maintain StumpWM, I think I remember your extension. Was yours the one to support multiple maildirs? Anyway you most certainly can limit yourself to a SBCL when working for StumpWM. We dropped support for anything but SBCL a couple of years ago. One of the reasons I was in favor of only supporting SBCL was precisely because we could use stuff in sb-posix to interact with the OS more easily.
I think it is a good example, precisely because the API in the CL standard for manipulating directories sucks, or at least it sucks if you are in UNIX. From reading the documentation to the POSIX API of guile it is similar to the one offered in sb-posix. But it shows what I was getting at: The standard only provides a ‘lower ceiling’ for the APIs offered by different implementations.
Exactly.
Oh, I thought that I had seen portable code in another extension, making me think that all code should be portable? Either way, even if it is possible, I always consider it a pity to use such extensions, even if it’s pragmatic.
It’s not just RMS’ disdain for CL. There are more people than RMS who prefer non-CL lisps to CL.
Yes, but I think he was a major force in keeping Emacs Lisp as different from Common Lisp as it is, as well as in deciding on Scheme for the official GNU extension language.