A level up the stack, I have slightly different perspective:
The pc platform is nothing short of miraculous. The existence of an open, broadly adopted standard that everyone can use to get something up and running is a boon, and it does not seem to have been replicated. From what I hear, arm is still a fragmentary shitshow, for instance. Sure, bioses have some fragmentation, but very little in practice, and efi is even more cohesive.
It is true, of course, that all abstractions have a cost. Joel’s law of software abstraction is universal. I think integration is a good idea, in general. But this is a bizarre target. I have never felt hampered in my ability to design systems by the firmware<->os bringup protocol. And I can’t speak to firmware authors, but I can’t imagine that supporting these interfaces is a tremendous burden for them either. Acpi, pci, usb, etc. are good enough, on the whole and for the most part—but even those would make better targets. If we are trying to co-design hardware and software—why are we sticking with decades-old isas? I want encodings tuned to empirically minimise code size. Cheap userspace interrupts, and read barriers built into my loads. Fine-grained bounds checks built into all my memory accesses.
A CPU business is not an easy thing to build—azul managed for a bit, but no longer sells computers, for instance—and I don’t mean to imply that, if one is interested in cohesion, one must build a CPU. But this seems like a very trivial target to take aim at, and the loss of portability does not seem worth its weight.
This is a bit of a surprising take; when you say that “you have never felt hampered” in your ability to design systems, what kind of systems are we talking about here? In particular: are we talking about server-side computers? And when you say you “can’t imagine that supporting these interfaces is a tremendous burden”, is that based on any experience with UEFI? Even UEFI’s proponents (such as there are any, as even the inventor of UEFI believes that it should be replaced!) would acknowledge that UEFI places tremendous constraints on firmware implementation.
From what I hear, arm is still a fragmentary shitshow, for instance. Sure, bioses have some fragmentation, but very little in practice, and efi is even more cohesive.
It isn’t that bad now (at least for systems that aim for the same space as x86 hardware), but mostly as a result of adopting UEFI.
A better comparison point would be IBM and Sun workstations (or Apple Macs) from the ’90s and early 2000s. These all used OpenFirmware, which was vastly better than UEFI. It had a lighweight Forth interpreter that allows the same ROM code for devices to execute on PowerPC or SPARC hardware. A lot of embedded devices use Flattened Device Trees (FDTs) for SoC device enumeration. FDT is a serialisation of the Device Tree format from OpenFirmware.
I suppose it depends how much value you put on being able to run supervisors not really designed for the hardware/firmware onto new hardware. So much of the PC is made weird by having to run older OSes on the newest HW, instead of being developed in near-lockstep.
I was just talking with a neighbor about this during school drop-off today. He works at IBM on the zSeries firmware team, and thought the idea of “holistic systems” sounded pretty familiar. :)
In all seriousness, though, I laud folks like Oxide, Talos, Arduino, Adafruit, et. al. in bringing top-to-bottom open designs to the consumer market. As someone fairly new to embedded development I’ve been pretty regularly surprised and disappointed by how much of the firmware development world assumes that a proprietary, Windows-only IDE and closed-source SDKs are just fine.
Linux seems to be pretty dominant now in the “embedded PC” world; here’s hoping Hubris, Zephyr, RIOT, or something in that ilk ends up becoming prevalent enough to let me stop worrying about whether the firmware code I’m developing will survive any change in vendor/toolchain.
Now that I’ve listened to the recording: Do I understand correctly that Oxide is storing the Helios kernel in SPI flash, then loading userland (and possibly loadable kernel modules) from NVMe flash? If so, is the SPI flash payload compressed? I imagine it could be.
One consequence of the boot picker being implemented as a macOS application behind the scenes is that is has full accessibility support (including VoiceOver), which is rather unique.
A level up the stack, I have slightly different perspective:
The pc platform is nothing short of miraculous. The existence of an open, broadly adopted standard that everyone can use to get something up and running is a boon, and it does not seem to have been replicated. From what I hear, arm is still a fragmentary shitshow, for instance. Sure, bioses have some fragmentation, but very little in practice, and efi is even more cohesive.
It is true, of course, that all abstractions have a cost. Joel’s law of software abstraction is universal. I think integration is a good idea, in general. But this is a bizarre target. I have never felt hampered in my ability to design systems by the firmware<->os bringup protocol. And I can’t speak to firmware authors, but I can’t imagine that supporting these interfaces is a tremendous burden for them either. Acpi, pci, usb, etc. are good enough, on the whole and for the most part—but even those would make better targets. If we are trying to co-design hardware and software—why are we sticking with decades-old isas? I want encodings tuned to empirically minimise code size. Cheap userspace interrupts, and read barriers built into my loads. Fine-grained bounds checks built into all my memory accesses.
A CPU business is not an easy thing to build—azul managed for a bit, but no longer sells computers, for instance—and I don’t mean to imply that, if one is interested in cohesion, one must build a CPU. But this seems like a very trivial target to take aim at, and the loss of portability does not seem worth its weight.
This is a bit of a surprising take; when you say that “you have never felt hampered” in your ability to design systems, what kind of systems are we talking about here? In particular: are we talking about server-side computers? And when you say you “can’t imagine that supporting these interfaces is a tremendous burden”, is that based on any experience with UEFI? Even UEFI’s proponents (such as there are any, as even the inventor of UEFI believes that it should be replaced!) would acknowledge that UEFI places tremendous constraints on firmware implementation.
It isn’t that bad now (at least for systems that aim for the same space as x86 hardware), but mostly as a result of adopting UEFI.
A better comparison point would be IBM and Sun workstations (or Apple Macs) from the ’90s and early 2000s. These all used OpenFirmware, which was vastly better than UEFI. It had a lighweight Forth interpreter that allows the same ROM code for devices to execute on PowerPC or SPARC hardware. A lot of embedded devices use Flattened Device Trees (FDTs) for SoC device enumeration. FDT is a serialisation of the Device Tree format from OpenFirmware.
Everyone I talked to at IBM despised writing drivers in Forth. To them, Petitboot was a major improvement.
I suppose it depends how much value you put on being able to run supervisors not really designed for the hardware/firmware onto new hardware. So much of the PC is made weird by having to run older OSes on the newest HW, instead of being developed in near-lockstep.
I was just talking with a neighbor about this during school drop-off today. He works at IBM on the zSeries firmware team, and thought the idea of “holistic systems” sounded pretty familiar. :)
In all seriousness, though, I laud folks like Oxide, Talos, Arduino, Adafruit, et. al. in bringing top-to-bottom open designs to the consumer market. As someone fairly new to embedded development I’ve been pretty regularly surprised and disappointed by how much of the firmware development world assumes that a proprietary, Windows-only IDE and closed-source SDKs are just fine.
Linux seems to be pretty dominant now in the “embedded PC” world; here’s hoping Hubris, Zephyr, RIOT, or something in that ilk ends up becoming prevalent enough to let me stop worrying about whether the firmware code I’m developing will survive any change in vendor/toolchain.
Looking forward to the video.
Not sure if you meant looking forward to watching the video or for it to be published. If the latter the slide deck links to https://www.youtube.com/watch?v=eNI0wFgBNmY#t=7044s
Now that I’ve listened to the recording: Do I understand correctly that Oxide is storing the Helios kernel in SPI flash, then loading userland (and possibly loadable kernel modules) from NVMe flash? If so, is the SPI flash payload compressed? I imagine it could be.
Does anything recent exist that can be considered a holistic system?
For personal computing, I would say that the Apple M1 is definitely a holistic system – and there are many instances of personal devices.
I agree about Apple Silicon Macs. Asahi Linux’s description of the Apple Silicon boot process is interesting. My favorite part:
Precursor comes close in the mobile space: https://www.crowdsupply.com/sutajio-kosagi/precursor