One day, while studying old code, I found out that it’s possible to encode
Windows Portable Executable files as a UNIX Sixth Edition shell script,
due to the fact that the Thompson Shell didn’t use a shebang line.
Once I realized it’s possible to create a synthesis of the binary formats being
used by Unix, Windows, and MacOS, I couldn’t resist the temptation of making it
a reality, since it means that high-performance native code can be almost as
pain-free as web apps. Here’s how it works:
That is really a Unix wart, and it can be a gotcha. You have to be
careful with commands like su that pass their arguments to the shell,
because you can inadvertently execute something in the shell that
wasn’t meant to be executed by the shell. As a long time Unix user,
I consider /bin/sh a frenemy due to its assorted warts and historical baggage,
and I wish we could make a clean break with it.
It’ll be nice to know that any normal PC program we write will “just work” on Raspberry Pi and Apple ARM. All we have to do embed an ARM build of the emulator above within our x86 executables, and have them morph and re-exec appropriately, similar to how Cosmopolitan is already doing doing with qemu-x86_64, except that this wouldn’t need to be installed beforehand.
I’d like to understand what Rosetta 2 would do with an APE binary that either does or does not bundle an emulator for ARM systems.
But just like the shell behavior advisory, that’s just a side question, and the potential benefits here are really appealing. I wonder if we’ll look back on this and feel like it was obvious in hindsight.
This is very cool, but it’s hard to understand how it works (after reading both posts) because I can’t keep up with the breakneck pace! I’m trying to figure out what the executable’s header looks like — how can it simultaneously have a MachO and ELF and whatever-Windows-uses magic number? — but after “here’s how it works:” she goes into some shell-script with, I guess, all-caps where various binary metadata would go? Is this a script that builds an executable, or the actual file itself? What’s with the redirection to file descriptor 7?
It’s clear I don’t know enough about sh or ELF or the Linux syscall ABI. I’m not sure Justine cares whether people slower than her can understand her ideas, but I’m curious anyway. Clues appreciated.
My guess (I haven’t looked deeply) is that the binary is either a PE executable (since NT will execute those even w/ a .COM extension). Maybe a DOS stub for Windows, but that won’t work on amd64. She mentions the misfeature of the Thompson shell, so I’m guessing the script will detect what platform it runs on and poop out a new executable as needed.
Tellingly, I don’t see much mention about ISA portability.
Replying to myself: it looks like the first few bytes are either interpreted as a no-op variable assignment by sh, or as raw x86 code by Windows. In the latter case those instructions jump to the embedded bootstrap code.
I’m still unclear what happens next on Unix. Somehow it needs to determine the OS and produce a binary starting with the appropriate header, but I don’t see where that happens.
Please note this is intended for people who don’t care about desktop GUIs
I wonder if you could write portable OpenGL applications on this. Could you use dlopen (or something) to detect the platform, and run whatever platform-specific initialization you need?
It ought to, unless for some reason Nim’s generated code makes assumptions about having nonstandard GCC lib calls available. Definitely worth a try.
I’m somewhat doubtful you could write nontrivial networking code this way, because there are subtle differences in how the socket I/O calls work between platforms, according to the texts I’ve read. This libc would need to include platform-specific shims for those.
This is probably the coolest hack I’ve seen all year!
If I understand it correctly, ARM compatibility uses magic-number tricks to make the system jump to an embedded x86 emulator. I wonder if it’d instead be possible to include both x86 and ARM versions of the compiled C code in the same file. Probably not doable with a single gcc invocation, but it’d give you bare-metal performance, which is kind of important if you’re targeting platforms like Raspberry Pi which aren’t going to emulate x86 code at blazing fast speed.
This is technically brilliant and an amazingly clever hack. Quoting from the referenced blog post about portable executables:
That is really a Unix wart, and it can be a gotcha. You have to be careful with commands like su that pass their arguments to the shell, because you can inadvertently execute something in the shell that wasn’t meant to be executed by the shell. As a long time Unix user, I consider /bin/sh a frenemy due to its assorted warts and historical baggage, and I wish we could make a clean break with it.
Anyway this is impressive.
Also quoting from the APE post,
I’d like to understand what Rosetta 2 would do with an APE binary that either does or does not bundle an emulator for ARM systems.
But just like the shell behavior advisory, that’s just a side question, and the potential benefits here are really appealing. I wonder if we’ll look back on this and feel like it was obvious in hindsight.
This is very cool, but it’s hard to understand how it works (after reading both posts) because I can’t keep up with the breakneck pace! I’m trying to figure out what the executable’s header looks like — how can it simultaneously have a MachO and ELF and whatever-Windows-uses magic number? — but after “here’s how it works:” she goes into some shell-script with, I guess, all-caps where various binary metadata would go? Is this a script that builds an executable, or the actual file itself? What’s with the redirection to file descriptor 7?
It’s clear I don’t know enough about sh or ELF or the Linux syscall ABI. I’m not sure Justine cares whether people slower than her can understand her ideas, but I’m curious anyway. Clues appreciated.
My guess (I haven’t looked deeply) is that the binary is either a PE executable (since NT will execute those even w/ a
.COMextension). Maybe a DOS stub for Windows, but that won’t work on amd64. She mentions the misfeature of the Thompson shell, so I’m guessing the script will detect what platform it runs on and poop out a new executable as needed.Tellingly, I don’t see much mention about ISA portability.
If the initial self exec fails, it attempts a self exec using the qemu-x86_64 user mode emulator.
Thanks, I hate it!
Replying to myself: it looks like the first few bytes are either interpreted as a no-op variable assignment by sh, or as raw x86 code by Windows. In the latter case those instructions jump to the embedded bootstrap code.
I’m still unclear what happens next on Unix. Somehow it needs to determine the OS and produce a binary starting with the appropriate header, but I don’t see where that happens.
On unix perhaps the default is to run with /bin/sh if nothing else can be determined, and most of the junk is just a noop when run. Just a guess.
This is possibly one of the most amazing low-level hacks I’ve ever seen. Unfortunately, I couldn’t get it running:
Debian 10, x86_64, dash 0.5.10.2-5.
Genode similarly supports the same user binaries (including drivers) across several kernels, as different as seL4 and Linux.
I wonder if you could write portable OpenGL applications on this. Could you use
dlopen(or something) to detect the platform, and run whatever platform-specific initialization you need?Does anyone have an idea if there’s a chance that compiling .c files emitted by Nim with this libc could work?
It ought to, unless for some reason Nim’s generated code makes assumptions about having nonstandard GCC lib calls available. Definitely worth a try.
I’m somewhat doubtful you could write nontrivial networking code this way, because there are subtle differences in how the socket I/O calls work between platforms, according to the texts I’ve read. This libc would need to include platform-specific shims for those.
I think it’d be possible to create an EFI stub in APE, too, in addition to the BIOS boot sector loader. After all, EFI applications are PE format…
This is probably the coolest hack I’ve seen all year!
If I understand it correctly, ARM compatibility uses magic-number tricks to make the system jump to an embedded x86 emulator. I wonder if it’d instead be possible to include both x86 and ARM versions of the compiled C code in the same file. Probably not doable with a single gcc invocation, but it’d give you bare-metal performance, which is kind of important if you’re targeting platforms like Raspberry Pi which aren’t going to emulate x86 code at blazing fast speed.