Threads for rsms

  1. 1

    What can be this used for? The readme makes it very clear how one can use it but I don’t understand what are the use-cases.

    1. 1

      I use zig for it atm (works on Windows, can cross compile, includes what is probably the best C++ build system available today) but you can commit the compiler and have an entirely self contained repo that works anywhere.

      There are some downsides: zig c++ doesn’t seem to generate debug info properly so we only use it for release builds, clang is the worst of the three major compilers, GitHub won’t let you push zig.exe because it’s too big so you have to do a gzip dance (which git does internally so this saves nothing for GitHub anyway), we precompile dependencies and commit those too and libc/libstdc++ mismatch BS is annoying, convincing third party build systems to use zig instead of the host compiler is annoying, …

      1. 1

        AFAIU, easier/simpler cross compiling. (x86 on arm or vice versa)

        1. 1

          But the README says “Currently the toolchain is not cross-compilation capable”. :(

          1. 1

            Ah. Good point I missed this.

            1. 1

              That’s right. But, we have a set of sysroots with libc’s ad hope to do what zig does: bundle libc & libc++ sources and build sysroots for “cross” targets on demand. It would be done with a dedicated tool (e.g. llvmbox-mksysroot) rather than integrated since llvmbox’s aim is to offer llvm vanilla; “as is.”

        1. 8

          Looks to be at least in part based on Jakub and my work on zig cc (original file).

          Curious that there is no mention of zig cc in the readme. I would have expected to see a “tradeoffs” section - why one might choose to use one project or the other depending on the use case.

          1. 3

            Hi! Happy to add a mention — we learned a lot from how you do it and gained even more respect for the hard work that has gone into Zig!

            1. 2

              To be clear, I don’t have any problem with what you are doing, and I am not asking for credit - I am merely curious what you are up to.

              1. 4

                We’re building a systems programming language called Zlug that is … JK. I’ve added some thoughts under “motivation” here https://github.com/rsms/llvmbox/blob/main/README.md#motivation — basically I’m working on some systems that doesn’t have glibc and no package managers (or even normal libc.) OS dev and some other projects. Often I need a compiler quickly on a blank system. This helps with that.

                The other part is building hobby compilers that embed clang & lld to get C & C++ compilation capabilities (in addition to LLVM codegen, which is of course the most common use of llvm as a library.) However, as you know very well, you’ll get into a pickle when the llvm libs are built and/or linked with a different flavor and/or version of libc and libc++ than your hobby compiler. llvmbox is—like zig—built against the exact same libc++ and libc that it ships with, allowing linking with the llvm libs into a third-party project built with those same libs. Key here is that libc and libc++ on Linux aren’t distro specific, so you can for example build your project on a ubuntu CI instance but run the result on an Alpine machine. Makes that story a lot simpler than if the third party project in this example would need to compile llvm for every Linux distro they want to support.

                Anyhow, I’m glad we got this figured out. I’ve worked with llvm in a number of projects over a few years and totally underestimated how much work and research had to be put in to make this (llvmbox) a reality. Chris and I got on Discord in the morning, exchanged findings from last night, then we’d go hammer some nails and bang some heads against some walls in different directions, syncing up again in the afternoon. It was fun but also pretty boring and frustrating to spend so much time on something that I really don’t want to do at all.

                I’ve tried my best to document the process (mostly in the shell scripts) and to keep the tooling for llvmbox itself as simple as possible (it’s basically just a collection of bash scripts. Yeah I like bash, fight me!)

                Hope it can benefit others!

          1. 2

            I’ve been using nix recently for building static versions of Dawn for Linux.

            I really enjoy the simplicity of the .nix expression language and definition files. Being into the idea of statically-compiled code I use musl and oh wow does it take a very long time to build/start a nix-shell. We’re talking ~1h on a 16 CPU Ryzen 3950x. But once it’s built, starting a nix-shell is reasonably quick (~1s.)

            The one big issue I’ve had with nix is that it uses the host machine as its base layer (I assume they use overlayfs) and it leaks through! I.e. the same exact .nix recipe run on two different machines may yield different outcomes, which “defies” a key selling point of nix. Someone pointed out to me that I could use buildFHSUserEnv but it does not seem to be compatible with musl or some of the packages I need (llvm 13, musl, cmake, ninja, python, x11).

            Anyhow, I find it a great alternative to Docker for build environments (especially since docker runs as root on Linux and does not do uid/gid mapping) and I’ll keep using nix I think.

            1. 2

              In what way do you mean it leaks through? I thought that as long as you use pinning/hashes/flakes your whole environment+versions is a part of the package definition. Or do you mean something like different CPUs result in different optimisation which results in different ordering / other details?

              1. 2

                “inside” the nix shell things like the host system’s /usr/lib directory is available. This caused issues for me with some build scripts that would go look in those “standard” locations and as a result link against libs from the host system. I believe the idea is that every nix tool e.g. gcc or cmake is built to only consider /nix/ stores but as soon as you run something that wasn’t patched for nix (in my case dawn and depot_tools) you start getting into trouble. Just gotta be conscious about it though and explicitly specify locations of things.