There’s some incredible engineering work that’s gone into this project, it’s been brilliant fun to follow along with the progress over the last few years.
Am I the only one that found this hilarious? The pointed references to “the manufacturer” got me laughing first. “It’s a little awkward that we beat the big corporation…” got me again. And then the subtle “the proprietary compiler won’t use it when compiling our test shaders” (which of course ended up being “but our compiler will”) got me a third time.
Eh, the phrasing makes it sound like Apple isn’t supporting it because they haven’t worked it out yet, rather than because they deliberately deprecated OpenGL on their platforms 5 years ago.
There is a clear implication here: they intend to implement vulkan. My understanding, from what raph levien has written, is that vulkan provides certain guarantees around forward progress for concurrent programs which metal does not; and metal does not provide these guarantees because the hardware does not. Assuming this is true, I wonder how they intend to tackle the problem.
EDIT: I was misremembering. The forward progress guarantee is something specifically offered by nvidia, but not by vulkan or other gpu manufacturers. What’s actually missing from metal is a certain barrier type. Which it seems at least more plausible you’d be able to slot in after the fact (though whether it’s actually possible, I still have no idea).
This is super impressive and I’ve been watching the Asahi project for a while, now.
And as a developer, I’ve noticed that I’ve been sort of mentally starting to leave Apple (as a longtime fan) for a number of reasons. One is the “iOS-ification” of macOS, complete with slowly encroaching garden walls, while I become more and more enamored of open systems. Another is the lack of AAA games (not a new thing, but when Blizzard, who used to release simultaneous Mac and PC versions of their games, stopped doing that at all with Overwatch and then with Diablo 4…).
The very first mainframes were best tested using a game, because it was the only code that tested every part of the system. It should be no surprise, then, that GPU’s, originally intended only for games, became crucial elements of new technologies all on their own (such as LLM’s). Games and approachable (read: “open”) coding have been the roots and veins of computing, and Apple seems to no longer be interested in this; I guess they imagine that every remaining Apple Mac user is a top-tier photographer or videographer, because musicians also seem to be leaving.
Games are also where the kids first go. And some of them discover almost by accident (maybe by trying to mod a game) that computers are actually really interesting on their own. And that begets the next generation of code builders. Apple’s not really nurturing this at all anymore. Linux is.
Perhaps that is because Tim Cook is the furthest thing from either a programmer, or a gamer. He may understand design, but he seems to be completely deaf to these other things.
The fact that Linux now supports far more games, and far better, than macOS, is hilarious. And as it turns out, well-deserved.
I have a 2 year old son, and I’m introducing him to Linux first at this point.
We want to set the right example for the ecosystem: the way forward is implementing open standards, conformant to the specifications, without compromises for “portability”.
Getting OpenGL® ES 3.1 to run on Asahi Linux is an achievement but mixing it up with normative statements like “the right example” and “the way forward” is a bit strange, for lack of a better term. It’s adding ideology into what is a pragmatic issue. Open standards aren’t inherently good for being open per se, the quality and the feasibility of the standard is what essentially matters. Apple has done relatively well without OpenGL® ES 3.1 support, despite OpenGL® ES 3.1 being open and Metal being closed.
Normative arguments in programming communities is not a new phenomenon. Zed Shaw had a recent talk about how this type of rhetoric operates in programming communities specifically.
The only reason Metal does well is because Apple is a locked down ecosystem.
Over time, people start to love and defend the prisons they are willingly caught up in, up to the point that they are ridiculing, questioning and ultimately despising those with the energy and commitment to break free from them.
Godspeed to Alyssa Rosenzweig and her team for this marvelous achievement!
The only reason Metal does well is because Apple is a locked down ecosystem.
There’s something of a history in graphics APIs of closed ones being superior. OpenGL didn’t lose out to DirectX because Microsoft pushed Direct3D, it lost because you could write Direct3D code that ran on half a dozen GPU makers’ cards and on Xbox, whereas the OpenGL code had to have a load of different paths for different spellings of the same thing via vendor extensions and even people like John Carmack, who had a lot of non-technical reasons for wanting to use OpenGL, used Direct3D and wrote rants about the Kronos group’s failure to make OpenGL competitive. Before that, 3dfx’s GLide was more popular for games than OpenGL, even though there was an OpenGL implementation available for their cards.
From what I’ve seen of Metal, it’s a fairly nice API that is only superficially different to Direct3D 12 or Vulkan in a handful of ways. There’s an implementation of Vulkan that runs on top of metal, so nothing is stopping you using Vulkan on Apple platforms, but it looks as if few people choose to do this if they don’t need portability, which suggests that Metal is doing something right.
Outside of the graphics arena, CUDA is vastly more popular for compute on GPUs than anything open.
The problem with open standards is that you need to please multiple vendors, each of which has some things that they do well and some less well than competitors. You either make a huge API that can do everyone’s fast things (but has no performance portability) or a small API that doesn’t expose all of the fast things on some hardware. With GLide and CUDA, 3dfx and NVIDIA just made APIs for their hardware and ignored everyone else. With Metal, Apple can make an API for what they want to do and co-design it with their GPUs so that they can avoid spending any silicon area or power budget on things that they don’t need. With Direct3D, Microsoft tries a bit to build consensus between the GPU vendors and then uses its dominant market position to say ‘this is what will be in the next version, if you can’t do it you can’t put the DirectX logo on your marketing, if you can’t do it fast then you’ll look bad in the benchmarks that we publish’. With OpenGL and Vulkan, there’s no one empowered to make the hard calls.
OpenGL didn’t lose out to DirectX because Microsoft pushed Direct3D, it lost because you could write Direct3D code that ran on half a dozen GPU makers’ cards and on Xbox, whereas the OpenGL code had to have a load of different paths for different spellings of the same thing via vendor extensions
For those interested in a full history of this time period, it’s covered in this epic StackOverflow answer.
I would also like to claim that there is a lot of general incompetence and lack of will to even try useful stuff among the vendors. Look at AMD’s current compute effort, ROCm and HIP: despite HIP being a straight clone of CUDA, the whole ROCm stack sucks and makes mistakes Nvidia managed to avoid 15 years ago. And this is AMD’s own system, that isn’t a collaboration with anyone. It’s no wonder OpenCL never went anywhere, with vendors like that.
A lot of people like to blame Nvidia for OpenCL 2.0’s failure, but neither AMD nor Intel ever actually produced functional OpenCL 2.0 drivers, they made partial and buggy drivers and then sorta just gave up.
(And indeed, Apple’s Metal was from the start intended as an OpenCL replacement just as much as an OpenGL replacement, hence Metal compute shaders are roughly equivalent in power and ease of use to CUDA, while Vulkan compute shaders are very limited.)
The only reason Metal does well is because Apple is a locked down ecosystem.
That is precisely my point. The openness of an API, or lack thereof, does not seem to critically affect the success, usefulness, or capability of an API.
People building and promoting systems often mention the ideology that drives them and claim that ideology as a feature of the system, not merely incidental.
For example in the west we go on about democracy and human rights but there are some very successful countries that don’t embrace those values. Personally I don’t want to live somewhere like China or Qatar, but I also don’t want to use macOS. I have chosen to run macOS in the past and I live in a country with worse human rights and democracy than where I grew up, so I’m not claiming any kind of ideological purity - just that ideology factors into my pragmatic choices.
just that ideology factors into my pragmatic choices.
Overall I thought your comparison of Apple to China or Qatar was an interesting point but this last statement missed some nuance in my comment. I believe you are characterizing the decision itself of where to live as a pragmatic choice whereas my comment was more about pragmatic or ideological decision-making. I would definitely concede that the decision of where to live is a personal one and just like you, the material conditions of China and Qatar wouldn’t automatically override my personal preferences. On the other hand, I believe personal preference is generally less relevant when choosing an API to base your application on. I will use Metal if I have to, I will use DirectX if I have to, both because one of my goals as a software developer is to make sure my software is as accessible as possible.
As an aside, I don’t think the decision to implement some version of OpenGL on the M1 was ideological at its core. I don’t think there was any other option for them really, outside of implementing some version of Vulkan, so it seemed more like a hard constraint.
I don’t think there was any other option for them really, outside of implementing some version of Vulkan, so it seemed more like a hard constraint.
Vulkan didn’t exist when they developed Metal, and once they’d started implementing all their things in Metal and teaching it to the developers in their ecosystem it’d be a very big investment wasted to switch to Vulkan. Also, although Vulkan is mostly a success now, that wasn’t certain to happen when Apple needed to decide on a course of action.
Additionally, Khronos standards have tended to be insufficient for the needs of developers in the past, with various vendor extensions being essentially required to run games. This seems to be happening again with Vulkan (AFAIK, being just a bystander in this). By using their own API, they’re not at the mercy of the whims and trends of the developer community to the same extent.
Does any of this work help with virtualised Linux / *BSD? I saw that Apple’s virtualisation frameworks include support for a PV device that, I believe, is a thin wrapper around Metal command queues. I’d love to have fast 3D in UTM.
I don’t think it would help much. To some extent some of it maybe, but ultimately you’re still at the mercy of whatever the host Metal driver does with the hardware, which isn’t necessarily what you want for OpenGL or Vulkan. I suspect some of what they’re doing may be roughly equivalent to software implementing features using compute shaders, in which case that could be done under Metal.
Maybe MoltenVK would be a reasonable basis for getting Vulkan to work on the PV device?
There’s some incredible engineering work that’s gone into this project, it’s been brilliant fun to follow along with the progress over the last few years.
Am I the only one that found this hilarious? The pointed references to “the manufacturer” got me laughing first. “It’s a little awkward that we beat the big corporation…” got me again. And then the subtle “the proprietary compiler won’t use it when compiling our test shaders” (which of course ended up being “but our compiler will”) got me a third time.
Eh, the phrasing makes it sound like Apple isn’t supporting it because they haven’t worked it out yet, rather than because they deliberately deprecated OpenGL on their platforms 5 years ago.
But you’d hope they’d at least sell an OpenGL dongle
There is a clear implication here: they intend to implement vulkan. My understanding, from what raph levien has written, is that vulkan provides certain guarantees around forward progress for concurrent programs which metal does not; and metal does not provide these guarantees because the hardware does not. Assuming this is true, I wonder how they intend to tackle the problem.
EDIT: I was misremembering. The forward progress guarantee is something specifically offered by nvidia, but not by vulkan or other gpu manufacturers. What’s actually missing from metal is a certain barrier type. Which it seems at least more plausible you’d be able to slot in after the fact (though whether it’s actually possible, I still have no idea).
This is super impressive and I’ve been watching the Asahi project for a while, now.
And as a developer, I’ve noticed that I’ve been sort of mentally starting to leave Apple (as a longtime fan) for a number of reasons. One is the “iOS-ification” of macOS, complete with slowly encroaching garden walls, while I become more and more enamored of open systems. Another is the lack of AAA games (not a new thing, but when Blizzard, who used to release simultaneous Mac and PC versions of their games, stopped doing that at all with Overwatch and then with Diablo 4…).
The very first mainframes were best tested using a game, because it was the only code that tested every part of the system. It should be no surprise, then, that GPU’s, originally intended only for games, became crucial elements of new technologies all on their own (such as LLM’s). Games and approachable (read: “open”) coding have been the roots and veins of computing, and Apple seems to no longer be interested in this; I guess they imagine that every remaining Apple Mac user is a top-tier photographer or videographer, because musicians also seem to be leaving.
Games are also where the kids first go. And some of them discover almost by accident (maybe by trying to mod a game) that computers are actually really interesting on their own. And that begets the next generation of code builders. Apple’s not really nurturing this at all anymore. Linux is.
Perhaps that is because Tim Cook is the furthest thing from either a programmer, or a gamer. He may understand design, but he seems to be completely deaf to these other things.
The fact that Linux now supports far more games, and far better, than macOS, is hilarious. And as it turns out, well-deserved.
I have a 2 year old son, and I’m introducing him to Linux first at this point.
Getting OpenGL® ES 3.1 to run on Asahi Linux is an achievement but mixing it up with normative statements like “the right example” and “the way forward” is a bit strange, for lack of a better term. It’s adding ideology into what is a pragmatic issue. Open standards aren’t inherently good for being open per se, the quality and the feasibility of the standard is what essentially matters. Apple has done relatively well without OpenGL® ES 3.1 support, despite OpenGL® ES 3.1 being open and Metal being closed.
Normative arguments in programming communities is not a new phenomenon. Zed Shaw had a recent talk about how this type of rhetoric operates in programming communities specifically.
The post is pretty explicitly a call to action though, right? This isn’t an ‘accident’, it’s fully intended. Nothing strange about it.
The only reason Metal does well is because Apple is a locked down ecosystem.
Over time, people start to love and defend the prisons they are willingly caught up in, up to the point that they are ridiculing, questioning and ultimately despising those with the energy and commitment to break free from them.
Godspeed to Alyssa Rosenzweig and her team for this marvelous achievement!
There’s something of a history in graphics APIs of closed ones being superior. OpenGL didn’t lose out to DirectX because Microsoft pushed Direct3D, it lost because you could write Direct3D code that ran on half a dozen GPU makers’ cards and on Xbox, whereas the OpenGL code had to have a load of different paths for different spellings of the same thing via vendor extensions and even people like John Carmack, who had a lot of non-technical reasons for wanting to use OpenGL, used Direct3D and wrote rants about the Kronos group’s failure to make OpenGL competitive. Before that, 3dfx’s GLide was more popular for games than OpenGL, even though there was an OpenGL implementation available for their cards.
From what I’ve seen of Metal, it’s a fairly nice API that is only superficially different to Direct3D 12 or Vulkan in a handful of ways. There’s an implementation of Vulkan that runs on top of metal, so nothing is stopping you using Vulkan on Apple platforms, but it looks as if few people choose to do this if they don’t need portability, which suggests that Metal is doing something right.
Outside of the graphics arena, CUDA is vastly more popular for compute on GPUs than anything open.
The problem with open standards is that you need to please multiple vendors, each of which has some things that they do well and some less well than competitors. You either make a huge API that can do everyone’s fast things (but has no performance portability) or a small API that doesn’t expose all of the fast things on some hardware. With GLide and CUDA, 3dfx and NVIDIA just made APIs for their hardware and ignored everyone else. With Metal, Apple can make an API for what they want to do and co-design it with their GPUs so that they can avoid spending any silicon area or power budget on things that they don’t need. With Direct3D, Microsoft tries a bit to build consensus between the GPU vendors and then uses its dominant market position to say ‘this is what will be in the next version, if you can’t do it you can’t put the DirectX logo on your marketing, if you can’t do it fast then you’ll look bad in the benchmarks that we publish’. With OpenGL and Vulkan, there’s no one empowered to make the hard calls.
For those interested in a full history of this time period, it’s covered in this epic StackOverflow answer.
I would also like to claim that there is a lot of general incompetence and lack of will to even try useful stuff among the vendors. Look at AMD’s current compute effort, ROCm and HIP: despite HIP being a straight clone of CUDA, the whole ROCm stack sucks and makes mistakes Nvidia managed to avoid 15 years ago. And this is AMD’s own system, that isn’t a collaboration with anyone. It’s no wonder OpenCL never went anywhere, with vendors like that.
A lot of people like to blame Nvidia for OpenCL 2.0’s failure, but neither AMD nor Intel ever actually produced functional OpenCL 2.0 drivers, they made partial and buggy drivers and then sorta just gave up.
(And indeed, Apple’s Metal was from the start intended as an OpenCL replacement just as much as an OpenGL replacement, hence Metal compute shaders are roughly equivalent in power and ease of use to CUDA, while Vulkan compute shaders are very limited.)
That is precisely my point. The openness of an API, or lack thereof, does not seem to critically affect the success, usefulness, or capability of an API.
People building and promoting systems often mention the ideology that drives them and claim that ideology as a feature of the system, not merely incidental.
For example in the west we go on about democracy and human rights but there are some very successful countries that don’t embrace those values. Personally I don’t want to live somewhere like China or Qatar, but I also don’t want to use macOS. I have chosen to run macOS in the past and I live in a country with worse human rights and democracy than where I grew up, so I’m not claiming any kind of ideological purity - just that ideology factors into my pragmatic choices.
Overall I thought your comparison of Apple to China or Qatar was an interesting point but this last statement missed some nuance in my comment. I believe you are characterizing the decision itself of where to live as a pragmatic choice whereas my comment was more about pragmatic or ideological decision-making. I would definitely concede that the decision of where to live is a personal one and just like you, the material conditions of China and Qatar wouldn’t automatically override my personal preferences. On the other hand, I believe personal preference is generally less relevant when choosing an API to base your application on. I will use Metal if I have to, I will use DirectX if I have to, both because one of my goals as a software developer is to make sure my software is as accessible as possible.
As an aside, I don’t think the decision to implement some version of OpenGL on the M1 was ideological at its core. I don’t think there was any other option for them really, outside of implementing some version of Vulkan, so it seemed more like a hard constraint.
Vulkan didn’t exist when they developed Metal, and once they’d started implementing all their things in Metal and teaching it to the developers in their ecosystem it’d be a very big investment wasted to switch to Vulkan. Also, although Vulkan is mostly a success now, that wasn’t certain to happen when Apple needed to decide on a course of action.
Additionally, Khronos standards have tended to be insufficient for the needs of developers in the past, with various vendor extensions being essentially required to run games. This seems to be happening again with Vulkan (AFAIK, being just a bystander in this). By using their own API, they’re not at the mercy of the whims and trends of the developer community to the same extent.
My comment was referring to the Asahi Linux team, not Apple.
Does any of this work help with virtualised Linux / *BSD? I saw that Apple’s virtualisation frameworks include support for a PV device that, I believe, is a thin wrapper around Metal command queues. I’d love to have fast 3D in UTM.
I don’t think it would help much. To some extent some of it maybe, but ultimately you’re still at the mercy of whatever the host Metal driver does with the hardware, which isn’t necessarily what you want for OpenGL or Vulkan. I suspect some of what they’re doing may be roughly equivalent to software implementing features using compute shaders, in which case that could be done under Metal.
Maybe MoltenVK would be a reasonable basis for getting Vulkan to work on the PV device?