Curious that there is no mention of zig cc in the readme. I would have expected to see a “tradeoffs” section - why one might choose to use one project or the other depending on the use case.
We’re building a systems programming language called Zlug that is … JK. I’ve added some thoughts under “motivation” here https://github.com/rsms/llvmbox/blob/main/README.md#motivation — basically I’m working on some systems that doesn’t have glibc and no package managers (or even normal libc.) OS dev and some other projects. Often I need a compiler quickly on a blank system. This helps with that.
The other part is building hobby compilers that embed clang & lld to get C & C++ compilation capabilities (in addition to LLVM codegen, which is of course the most common use of llvm as a library.) However, as you know very well, you’ll get into a pickle when the llvm libs are built and/or linked with a different flavor and/or version of libc and libc++ than your hobby compiler. llvmbox is—like zig—built against the exact same libc++ and libc that it ships with, allowing linking with the llvm libs into a third-party project built with those same libs. Key here is that libc and libc++ on Linux aren’t distro specific, so you can for example build your project on a ubuntu CI instance but run the result on an Alpine machine. Makes that story a lot simpler than if the third party project in this example would need to compile llvm for every Linux distro they want to support.
Anyhow, I’m glad we got this figured out. I’ve worked with llvm in a number of projects over a few years and totally underestimated how much work and research had to be put in to make this (llvmbox) a reality. Chris and I got on Discord in the morning, exchanged findings from last night, then we’d go hammer some nails and bang some heads against some walls in different directions, syncing up again in the afternoon. It was fun but also pretty boring and frustrating to spend so much time on something that I really don’t want to do at all.
I’ve tried my best to document the process (mostly in the shell scripts) and to keep the tooling for llvmbox itself as simple as possible (it’s basically just a collection of bash scripts. Yeah I like bash, fight me!)
I use zig for it atm (works on Windows, can cross compile, includes what is probably the best C++ build system available today) but you can commit the compiler and have an entirely self contained repo that works anywhere.
There are some downsides: zig c++ doesn’t seem to generate debug info properly so we only use it for release builds, clang is the worst of the three major compilers, GitHub won’t let you push zig.exe because it’s too big so you have to do a gzip dance (which git does internally so this saves nothing for GitHub anyway), we precompile dependencies and commit those too and libc/libstdc++ mismatch BS is annoying, convincing third party build systems to use zig instead of the host compiler is annoying, …
That’s right. But, we have a set of sysroots with libc’s ad hope to do what zig does: bundle libc & libc++ sources and build sysroots for “cross” targets on demand. It would be done with a dedicated tool (e.g. llvmbox-mksysroot) rather than integrated since llvmbox’s aim is to offer llvm vanilla; “as is.”
Looks to be at least in part based on Jakub and my work on zig cc (original file).
Curious that there is no mention of zig cc in the readme. I would have expected to see a “tradeoffs” section - why one might choose to use one project or the other depending on the use case.
Hi! Happy to add a mention — we learned a lot from how you do it and gained even more respect for the hard work that has gone into Zig!
To be clear, I don’t have any problem with what you are doing, and I am not asking for credit - I am merely curious what you are up to.
We’re building a systems programming language called Zlug that is … JK. I’ve added some thoughts under “motivation” here https://github.com/rsms/llvmbox/blob/main/README.md#motivation — basically I’m working on some systems that doesn’t have glibc and no package managers (or even normal libc.) OS dev and some other projects. Often I need a compiler quickly on a blank system. This helps with that.
The other part is building hobby compilers that embed clang & lld to get C & C++ compilation capabilities (in addition to LLVM codegen, which is of course the most common use of llvm as a library.) However, as you know very well, you’ll get into a pickle when the llvm libs are built and/or linked with a different flavor and/or version of libc and libc++ than your hobby compiler. llvmbox is—like zig—built against the exact same libc++ and libc that it ships with, allowing linking with the llvm libs into a third-party project built with those same libs. Key here is that libc and libc++ on Linux aren’t distro specific, so you can for example build your project on a ubuntu CI instance but run the result on an Alpine machine. Makes that story a lot simpler than if the third party project in this example would need to compile llvm for every Linux distro they want to support.
Anyhow, I’m glad we got this figured out. I’ve worked with llvm in a number of projects over a few years and totally underestimated how much work and research had to be put in to make this (llvmbox) a reality. Chris and I got on Discord in the morning, exchanged findings from last night, then we’d go hammer some nails and bang some heads against some walls in different directions, syncing up again in the afternoon. It was fun but also pretty boring and frustrating to spend so much time on something that I really don’t want to do at all.
I’ve tried my best to document the process (mostly in the shell scripts) and to keep the tooling for llvmbox itself as simple as possible (it’s basically just a collection of bash scripts. Yeah I like bash, fight me!)
Hope it can benefit others!
What can be this used for? The readme makes it very clear how one can use it but I don’t understand what are the use-cases.
I use zig for it atm (works on Windows, can cross compile, includes what is probably the best C++ build system available today) but you can commit the compiler and have an entirely self contained repo that works anywhere.
There are some downsides:
zig c++doesn’t seem to generate debug info properly so we only use it for release builds, clang is the worst of the three major compilers, GitHub won’t let you push zig.exe because it’s too big so you have to do a gzip dance (which git does internally so this saves nothing for GitHub anyway), we precompile dependencies and commit those too and libc/libstdc++ mismatch BS is annoying, convincing third party build systems to use zig instead of the host compiler is annoying, …AFAIU, easier/simpler cross compiling. (x86 on arm or vice versa)
But the README says “Currently the toolchain is not cross-compilation capable”. :(
Ah. Good point I missed this.
That’s right. But, we have a set of sysroots with libc’s ad hope to do what zig does: bundle libc & libc++ sources and build sysroots for “cross” targets on demand. It would be done with a dedicated tool (e.g. llvmbox-mksysroot) rather than integrated since llvmbox’s aim is to offer llvm vanilla; “as is.”
Thank you!