Really impressed at someone able to deftly dig through LLVM in particular, I feel like I would have gotten to the IR stuff and just thrown it over the fence to someone else to figure out.
Very nice debugging story. I love reading through these kinds of notes.
I’d love to know just how long this took. Do I also correctly assume the author had a pretty good understanding of rustc or the LLVM toolchain before?
luqmana made >300 commits to rust-lang/rust (see here), so yes.
Great job, though I was secretly rooting for a memory corruption error, but it turned out to be an invalid llvm intermediate representation generation error in rustc.
This is what scares me a little with rust compared to c++: cargo hides all the compilation steps. The amount of smuggling of command parameters needed in the investigation here is astounding. How is a novice ever going to get a firm grasp of how compilation of the different units work, or how they get linked and optimized?
cargo -v prints out the rustc commands that cargo calls.
More than that though, it’s just “one rustc call per crate”, because rust’s unit of compilation is a crate (i.e. a library) not a file. The rustc calls are not where any the complexity lies.
All the “smuggling” you see going on in this investigation isn’t between cargo and rustc, it’s between rustc and llvm, and it’s presumably equivalent to smuggling that goes on beyond clang and llvm. You don’t see it with C either.
One of the biggest reasons you don’t see the lower stages of C compilation is because they have been battle tested for decades and rarely misperform with this level of disruption. Rust is about 11 years old. It’s been about 6 years since the 1.0 release. At the analogous point in C’s development it was still common to have the compiler output assembler that was separately fed into an assembler program. That was partially because portability didn’t carry the same force as it eventually would. It was also because the assemblers were still maturing and would need to be tweaked, replaced, upgraded, etc. I was still expected to learn how to do this when I was learning C in the early-mid 90s. It was front-and-center in the C compiler docs at that time.
This is the stage I’m at with it. I bought some ATtiny chips and tried using rust with them. There’s some similarly oblique errors with the AVR backend (related to register allocation IIRC). I’d love to be helpful in fixing them but I spent 2 hours tracking down the relevant bugs and then I was out of time for delving down to the next level. If it had been nested Makefiles I would have seen the commands for each level of compilation much sooner.