1. 5

  2. 2

    So I was wondering whether this is actually necessary, https://www.forrestthewoods.com/blog/benchmarking-malloc-with-doom3/ suggests maybe not. OTOH some googling brought me to https://sound.stackexchange.com/questions/28163/whats-the-shortest-sound-perceptible-to-the-human-ear which suggests even super short sounds are audible (much shorter time period than visible motion, interestingly, movies maintain illusion of motion with changes only every 30ms), though I haven’t done the suggested experiment in one of the answers.

    A slow free() would lead to a super-short silence, which isn’t the same thing, but still.

    I’d be interested in hearing from someone who has some real-world experience: are the delays actually noticeable?

    1. 3

      Computers output audio by producing a steady stream of samples at a constant sample rate (usually 44.1kHz or 48kHz), and operating systems generally ask audio applications for fixed-size blocks of samples (e.g. 512 or 1024 samples) on a regular schedule. This means that by the time an audio application is writing out a sample value to a given location in the output buffer, the exact time at which that sample will be played back by the output device is essentially already scheduled.

      If the application finishes computing its samples on time, precisely how long it took to compute those samples has absolutely no effect on the timing of what the user hears. If it doesn’t finish on time, the result is a buffer underrun, and the user will hear an audible glitch. The precise nature of the glitch depends on what happens downstream in the pipeline (e.g. the user might hear a few milliseconds of silence, or maybe they’ll hear a repeat of the previous few millseconds, or maybe even garbage data), but it is generally extremely obvious when this happens.

      So the point of avoiding operations with unpredictable latency is not to ensure more consistent timing in the sounds that the user hears; the whole system is already playing back samples with extremely regular timing. The point is to avoid missing a deadline entirely, which results in an objectionable and extremely noticeable artifact.

      There’s a well known blog post by Ross Bencina called Time Waits for Nothing which goes into all of this in more depth, and is probably what first introduced a lot of people to the “don’t (de)allocate on the audio thread” dogma. The post was written in the context of real-time music software, where it’s simultaneously important to have very low latency (to enable performers to hear the results of their actions without noticeable delay) and high reliability (so you don’t ruin a live performance or studio recording with glitches). The article specifically mentions a figure of ~8ms end-to-end system latency, which requires software to be working with buffer sizes on the order of ~1ms. In that “Benchmarking Malloc with Doom 3” post, the author found the CRT malloc to have a p99.99 latency on the order of ~25us. With a 1ms buffer size, that means that if you call malloc 40 times per buffer, you can expect an audible glitch every 10 seconds on average. Pretty bad. In contrast, games and other audio applications can afford to work with larger buffer sizes and end-to-end latencies, so they can correspondingly get away with more variance in processing time without risking underruns.

      The Doom 3 article doesn’t really contradict anything I’m saying here; the takeaway is basically that there are certainly some audio applications where you can afford a bit of allocator usage on the audio thread, but there are other audio applications where it’s basically unacceptable.

      1. 1

        Thank you, this was very educational.

      2. 2

        I was thinking of it more as a substitute for garbage collection than a substitute for malloc. GC can be perfectly reasonably fast in terms of throughput, but latency is still a pain in the ass.

        My (limited) experience is that if you miss your deadline in sound code you don’t just get a pause, you get a pop or crackle. It sucks. Example here: https://alopex.li/temp/framedrop.flac