This one scratches only the surface of many many issues relating to digital video and audio. The <video> tag is a mess partly because of timestamp issues like this one. It makes me think the W3C doesn’t have many broadcast people on board.
The W3C did make an effort. The video and audio tags were not standardized until HTML5. At that time, W3C wanted to make choices in harmony with five major browser vendors: Apple, Google, Microsoft, Mozilla, and Opera. I recall that three of five vendors were required to agree in order to ratify anything, and they couldn’t agree on which formats to allow in multimedia tags; Microsoft refused to commit to anything, Apple and Google only signed off on formats in their respective patent pools, and Mozilla and Opera only signed off on open formats. They couldn’t even agree on PNG, if I recall correctly! (WP memorializes the discussions for video and audio tags respectively.)
Hah, yes, I remember these discussions on some mailing lists I’m on. At one point someone suggested H.261 as a common format since any patents on it are long expired. But the codec issue is not so much an issue now. What is an issue is timestamps. For some reason it was decided that timestamps should be floats. But with multimedia timestamps are always fractional numbers. Because of this there’s no reliable way too seek to a specific frame that works in every browser.
Another issue is if the video or audio doesn’t start at t=0 exactly. Audio starting before t=0 is particularly problematic, and is also very common for files in the wild since that is how ffmpeg deals with MDCT-ish audio codecs.