1. 52
  1. 7

    Let’s hope they don’t drown in yak shaving the new tooling, while actually trying to shave the yak called operating system.

    1. 7

      “created a language for my OS” is the new “created a game/rendering engine for my game” in terms of yak stacks.

    2. 11

      Generating C++ is an underrated strategy! C++ has great debugging and performance tools like uftrace, perf, CLion, etc. And you can link with anything.

      It can be annoying to write, e.g. writing all the redundant headers, but generating code from a different language solves that.

      Oil is doing something similar: Brief Descriptions of a Python to C++ Translator

      It gives us much better performance, as well as memory safety. There were several reactions like “Why C++? It’s not memory safe”. But the idea is to generate code with that property, as SerenityOS is doing. If you never write free() or delete in your source language, then there are no double-free / double-delete bugs.

      (The boundary between manually written C++ and generated C++ is still an issue .. I’ll be interested see their API. But it should be nicer and more direct than the interface between two totally separate languages.)

      There are also some interesting ways to generate fast code with C++ templates: https://souffle-lang.github.io/translate

      i.e. you get a fairly nice language for writing a templatized “runtime”, and then you generate very high level typed code that targets it. Then the C++ compiler mushes it all together into fast code.

      Oil’s scheme is simpler, but it has this flavor. The garbage collector and templated types interact in an interesting way, e.g. consider tracing Dict<int, int> vs. Dict<int, Str*>. (I’m working on a blog post about this.)

      1. 5

        There were several reactions like “Why C++? It’s not memory safe”.

        As if assembly directly were any safer.

        1. 3

          D with the -betterC mode could also be a good target for transpilers. It’s got most of the good stuff from C++ and the imperative/inspection-based meta programming model could be easier to target, I think. Also has 3 compilers and all of them compile faster than C++ (no preprocessor etc). The only problem in Serenity’s case would be binding to existing C++ code seamlessly.

          1. 3

            Yeah D is nice because it’s garbage collected already! It is a shame that people seem to shy away for certain things because it is garbage collected, but I view that as a big benefit.

            I’m guessing SerenityOS has a good reason for ARC though … So yeah that is a problem – GC’s are hard to tune and aren’t universal across applications. There was a thread here recently with Chris Lattner on GC vs ARC.

            I think Nim lets you choose the allocator, but to me a big downside is that the generated code isn’t readable. It is a huge benefit to be able to read it, step through it in the debugger, pretty print data structures, etc.

            1. 7

              The -betterC subset is not GCed. It doesn’t have a runtime. However, the good stuff (RAII, meta-programming) is all there… it can be tedious to use by hand but as a transpiler target I think it is a good choice.

          2. 1

            I generate C++ from Go in https://github.com/nikki93/gx and when I do multiple TUs I’m planning to actually just generated decls for all referenced symbols on the top, rather than generate #includes. This way each TU is much smaller.

            1. 1

              Generating C++ is an underrated strategy

              I have a suspicion that the reason it’s underrated is that common old wisdom says “since I’m going to all the effort to generate code at all, maybe I generate C instead to get faster compile times.”

              I wonder to what extent that is no longer relevant? My belief is that there’s not an interesting difference now because most time goes into optimisers rather than parsing.

              C++ totally makes sense for this context of course because they already have a C++ codebase which they want to link with. :)

              1. 3

                Well I definitely ran into some template bloat problems, so it’s a double-edged sword. I think it basically comes down to whether you want to use templates, classes, and exceptions, etc. My post mentions that:


                Templates increase compile times but they can generate faster code, as the Souffle Datalog example shows. I would think of it as a very expressive and high level language for monomorphization.

                Also, when you generate code you have more control over the translation units, so the nonlinear explosion in compile times can be mitigated. Right now Oil has 24 translation units, and I see the redundancy in build times. I even preprocessed each one and counted the 10x blowup in line count:

                https://oilshell.zulipchat.com/#narrow/stream/121539-oil-dev/topic/Preprocessed.20Output.20.28ninja.20is.20fun.29 (requires login)

                So less than 100K lines of code blows up to about 1M once all the template headers are included. But I could probably cut it down to 6 or 8 translation units, and compile a linear amount of code. I think it would end up at 200K or less.

                But yeah the point is that once you generate code you can do stuff like that, whereas manually writing C++ you often get “stuck”. Most big C++ projects have bad build systems and bad code structure …

                1. 2

                  Thanks for weighing in. <3

                  Templates increase compile times but

                  I was kinda thinking you wouldn’t use them in a code generator, since you can generate exactly the needed monomorphic versions yourself? but that might not take advantage of the C++ compiler’s ability to coalesce identical implementations sometimes.

                  Also, when you generate code you have more control over the translation units…

                  I was thinking that perhaps it might be possible to generate code with absolutely minimal headers / forward declarations too, which could help some more?

                  1. 2

                    Yeah I think it’s mainly about the implementation effort … Trying to recreate what C++ is doing could be the difference between a 1 year project and a 5 year one. I should know :)

                    e.g. I liken it to one of the “wrong turns” I mentioned on the blog – to “just” write a Python bytecode interpreter. Well it turns out that is a huge project, and there’s no way to make it fast with a reasonable amount of effort.

                    You can get something slow working, but getting good performance is a whole different story!

            2. 1

              This is one of the features that kicked really hard the AMIGA Workbench OS … because it lacked it.

              1. 1

                “the current landscape of memory-safe systems languages”

                I hadn’t realized there were more to this than Rust.

                1. -1

                  Jakt: immutable but object-oriented

                  2 steps forward and 1 step back.

                  1. 17

                    I don’t think anyone has successfully developed a large scale UI system in anything but OO. SerenityOS leans heavily on OO widgets for the UI, so I don’t think they have much choice.

                    1. 1

                      In what way?