Let’s pretend that JPEG has finally died (again) and GIF and PNG are both gone.
To me, PNG hits the perfect sweet spot for lossless images. It has decent compression, speed and is supported everywhere. It’s also relatively low complexity. The format has grown on me, even though significantly better lossless compression exists. So I’d be sad about its disappearance.
PNG is perfectly fine, as is JPEG. In fact, if we actually implemented the entire JPEG spec that would be pretty neat. Hierarchical coding is in there, which is key for coding large pictures efficiently. I am highly skeptical of introducing new formats when web people can’t even use JPEG correctly.
The only thing really missing is a format that can do lossy compression with transparency. You can kinda hack it with PNG, quantizing color values, but it’s not great.
WebP/WebP2—Baidu image format with lossy compression based on Baidu VPx codec (VP8 or VP10) and lossless compression from the French division of Baidu.
They’ve tended to pick a patented winner and avoid other codecs–they picked H.264, now H.265 and its relative .heic. But they did join AOM and eventually supported VP8 (called for by WebRTC), WebP (“free” with VP8), and VP9 (for 4kp30 YouTube video on Apple TV). The slow motion in the right direction is is encouraging, but haven’t seen any explicit public statements about AV1 or AVIF still.
I’m not bothered by multiple formats existing–AVIF seems to work well at low bitrates, JXL has some cool features with JPEG transcoding and lossless, HEIF (given its patent situation) can hopefully not be my problem. It is a little sad, though, if Apple will only support HEIF but other players will never support HEIF because of the cost.
That might mean the “next-gen lossy codec” we settle on is usually JPEG1, transcoded to JXL for clients that support it, perhaps with AVIF and WebP opportunistically sprinkled in the way WebP is today, along with HEIF if you’re rich and serve a lot of Safari users. I guess that’s somewhat better than today, especially if you’re large enough to really need to worry about image size, but unfortunate compared to what could be.
I think video and image are quite different use cases. For video, Apple really cares about power consumption, because video playback is very processor intensive. Supporting a small number of formats lets them carefully optimise fixed-function decoders for those formats in their mobile SoCs. If, for example, YouTube, has to ship videos in the format that is most battery efficient for iPhones because it’s the only format that iPhones will play from the web browser, then iPhones look good on battery-life benchmarks.
For images, the same constraint may apply for the formats used in the GUI, where you want to be able to put them easily in compressed textures and it might make sense to ship them in the compressed format that your GPU supports but that sounds like a stretch.
Interesting. The fixed function encode/decode hardware thing seems especially relevant for e.g. the H.264 decision made when you just couldn’t cram as much on the chip. On the other hand, HEVC and AV1 are both way trickier codecs moving more pixels, so maybe hardware has had to run just to stay in place.
Definitely lower requirements for one ‘keyframe’ image than a real video encode. AV1 and HEVC intra both seem heavy enough that hardware can make a real difference even for still images. JXL and/or WebP2 may end up light enough to be more tolerable in software.
Apple’s also mentioned worries about submarine patents as a reason for not supporting formats. AOM seems to have done about as much as can be done about them (building on existing codecs, member patents, legal review, defense fund) and JXL/VPx/WebP have tried to be royalty-free too (and VPx/WebP seem to’ve succeeded?).
An optimistic thought is that maybe the VP8/9 support really is a sign, and in a gen or two Apple bakes AV1 en/decode into their chips, and then AVIF comes ‘free’.
We’ll never know, but I wonder if there’d’ve been adoption of a WebP VP8->VP9 update when VP9 came out. VP9 seemed like enough of a step up to more often justify the work server side to handle an extra format, and Apple did (finally) accept both formats.
To me, PNG hits the perfect sweet spot for lossless images. It has decent compression, speed and is supported everywhere. It’s also relatively low complexity. The format has grown on me, even though significantly better lossless compression exists. So I’d be sad about its disappearance.
PNG is perfectly fine, as is JPEG. In fact, if we actually implemented the entire JPEG spec that would be pretty neat. Hierarchical coding is in there, which is key for coding large pictures efficiently. I am highly skeptical of introducing new formats when web people can’t even use JPEG correctly.
The only thing really missing is a format that can do lossy compression with transparency. You can kinda hack it with PNG, quantizing color values, but it’s not great.
Did the author s/Google/Baidu/g in this article?
I think they’re just grinding an axe.
I wonder what Apple’s gonna do.
They’ve tended to pick a patented winner and avoid other codecs–they picked H.264, now H.265 and its relative .heic. But they did join AOM and eventually supported VP8 (called for by WebRTC), WebP (“free” with VP8), and VP9 (for 4kp30 YouTube video on Apple TV). The slow motion in the right direction is is encouraging, but haven’t seen any explicit public statements about AV1 or AVIF still.
I’m not bothered by multiple formats existing–AVIF seems to work well at low bitrates, JXL has some cool features with JPEG transcoding and lossless, HEIF (given its patent situation) can hopefully not be my problem. It is a little sad, though, if Apple will only support HEIF but other players will never support HEIF because of the cost.
That might mean the “next-gen lossy codec” we settle on is usually JPEG1, transcoded to JXL for clients that support it, perhaps with AVIF and WebP opportunistically sprinkled in the way WebP is today, along with HEIF if you’re rich and serve a lot of Safari users. I guess that’s somewhat better than today, especially if you’re large enough to really need to worry about image size, but unfortunate compared to what could be.
I think video and image are quite different use cases. For video, Apple really cares about power consumption, because video playback is very processor intensive. Supporting a small number of formats lets them carefully optimise fixed-function decoders for those formats in their mobile SoCs. If, for example, YouTube, has to ship videos in the format that is most battery efficient for iPhones because it’s the only format that iPhones will play from the web browser, then iPhones look good on battery-life benchmarks.
For images, the same constraint may apply for the formats used in the GUI, where you want to be able to put them easily in compressed textures and it might make sense to ship them in the compressed format that your GPU supports but that sounds like a stretch.
Interesting. The fixed function encode/decode hardware thing seems especially relevant for e.g. the H.264 decision made when you just couldn’t cram as much on the chip. On the other hand, HEVC and AV1 are both way trickier codecs moving more pixels, so maybe hardware has had to run just to stay in place.
Definitely lower requirements for one ‘keyframe’ image than a real video encode. AV1 and HEVC intra both seem heavy enough that hardware can make a real difference even for still images. JXL and/or WebP2 may end up light enough to be more tolerable in software.
Apple’s also mentioned worries about submarine patents as a reason for not supporting formats. AOM seems to have done about as much as can be done about them (building on existing codecs, member patents, legal review, defense fund) and JXL/VPx/WebP have tried to be royalty-free too (and VPx/WebP seem to’ve succeeded?).
An optimistic thought is that maybe the VP8/9 support really is a sign, and in a gen or two Apple bakes AV1 en/decode into their chips, and then AVIF comes ‘free’.
We’ll never know, but I wonder if there’d’ve been adoption of a WebP VP8->VP9 update when VP9 came out. VP9 seemed like enough of a step up to more often justify the work server side to handle an extra format, and Apple did (finally) accept both formats.