Honestly, the sentiment isn’t bad, but AddressSanitizer gives you most of the benefit with a fraction of the runtime cost. I prefer to run tests with ASAN enabled and rarely use valgrind these days.
I prefer ASan for unit tests in CI (it’s sufficiently fast that I can run the entire test suite with it) but when I do get an ASan failure the output from Valgrind is usually a lot more informative for debugging. There’s also an important category of bug that Valgrind will catch but ASan won’t: ones where you have some subtle undefined behaviour in your program that the compiler exploits at higher optimisation levels. ASan annotations will often block these optimisations, whereas valgrind can run on the release build. The same applies to anything compiled with my nemesis, gcc 9, where valgrind will point to this week’s exciting miscompilation (I can’t wait for the Ubuntu LTS release that included GCC 9 to be EOL, it’s the buggiest compiler I’ve had to support as a target for at least 15 years).
I always cringe a bit when I see people using cachegrind though. Cachegrind does not measure your cache state, it runs your memory trace through a model of a cache that was fairly accurate in the mid ‘90s and now behaves nothing like a modern cache. It’s fine for coarse-grained ‘oh, you’re doing a load of pointer chasing here, that might be slow’ things but it’s unlikely to give a good performance model for a modern CPU.
Cachegrind does not measure your cache state, it runs your memory trace through a model of a cache that was fairly accurate in the mid ‘90s and now behaves nothing like a modern cache.
The article also mentions this fact, and I found it interesting. My interest in the tool is mostly rooted in curiosity & personal research rather than any serious work, but I just recently perused cachegrind’s docs — while they’re very explicit on the metrics being a result of a simulation (and a simplified one at that), I failed to perceive it as being very innacurate or obsolete. Good to know.
Not good ones. Modern CPUs have quite a rich set of cache performance counters but they are not very fine grained and the thing you really want is to know the top 10 instruction pointer values that trigger cache misses. The only way of getting that is with off-core debugging (Arm’s CoreSight is nice for this, Intel has a similar thing, but they’re very proprietary).
There is an large but not full overlap between the two. There are curious corner cases that asan catches that valgrind doesn’t and vice versa.
So after fiddling around and testing for awhile I ended up leaving valgrind for our main build as it seem to catch a larger set of our bugs, but periodically I run asan/ubsan.
I also enabled -fsanitize=bounds-strict -fsanitize-undefined-trap-on-error which catches most of the cases valgrind doesn’t.
An amazing history of an amazing set of tools, with a finishing line where I realize I’ve been mispronouncing it for years. Cheers to Valgrind (pronounced correctly now), and cheers to Julian, Nick, and everyone else involved in making it all work!
Such a great tool. Slightly eclipsed by ASAN these days, but it’s still vitally necessary to test your release builds under valgrind because it will catch things that ASAN builds can’t.
Note to newbies…
If you’re writing C/C++ (or similar language) and NOT using valgrind… stop whatever you’re doing now, and wire valgrind into your unit tests.
NOW.
Honestly, the sentiment isn’t bad, but AddressSanitizer gives you most of the benefit with a fraction of the runtime cost. I prefer to run tests with ASAN enabled and rarely use valgrind these days.
I prefer ASan for unit tests in CI (it’s sufficiently fast that I can run the entire test suite with it) but when I do get an ASan failure the output from Valgrind is usually a lot more informative for debugging. There’s also an important category of bug that Valgrind will catch but ASan won’t: ones where you have some subtle undefined behaviour in your program that the compiler exploits at higher optimisation levels. ASan annotations will often block these optimisations, whereas valgrind can run on the release build. The same applies to anything compiled with my nemesis, gcc 9, where valgrind will point to this week’s exciting miscompilation (I can’t wait for the Ubuntu LTS release that included GCC 9 to be EOL, it’s the buggiest compiler I’ve had to support as a target for at least 15 years).
I always cringe a bit when I see people using cachegrind though. Cachegrind does not measure your cache state, it runs your memory trace through a model of a cache that was fairly accurate in the mid ‘90s and now behaves nothing like a modern cache. It’s fine for coarse-grained ‘oh, you’re doing a load of pointer chasing here, that might be slow’ things but it’s unlikely to give a good performance model for a modern CPU.
The article also mentions this fact, and I found it interesting. My interest in the tool is mostly rooted in curiosity & personal research rather than any serious work, but I just recently perused cachegrind’s docs — while they’re very explicit on the metrics being a result of a simulation (and a simplified one at that), I failed to perceive it as being very innacurate or obsolete. Good to know.
Are there any alternatives worth exploring?
Not good ones. Modern CPUs have quite a rich set of cache performance counters but they are not very fine grained and the thing you really want is to know the top 10 instruction pointer values that trigger cache misses. The only way of getting that is with off-core debugging (Arm’s CoreSight is nice for this, Intel has a similar thing, but they’re very proprietary).
There is an large but not full overlap between the two. There are curious corner cases that asan catches that valgrind doesn’t and vice versa.
So after fiddling around and testing for awhile I ended up leaving valgrind for our main build as it seem to catch a larger set of our bugs, but periodically I run asan/ubsan.
I also enabled -fsanitize=bounds-strict -fsanitize-undefined-trap-on-error which catches most of the cases valgrind doesn’t.
Similar note to the advanced – if you have any systems with custom memory managers, teach valgrind how to understand them! It’s not hard. There is a reasonable one here: https://developers.redhat.com/articles/2022/03/23/use-valgrind-memcheck-custom-memory-manager , although I did things a bit differently myself. Should write about that.
An amazing history of an amazing set of tools, with a finishing line where I realize I’ve been mispronouncing it for years. Cheers to Valgrind (pronounced correctly now), and cheers to Julian, Nick, and everyone else involved in making it all work!
Valgrind is a nice tool, but AddressSanitizer is 10 times faster than Valgrind.
Such a great tool. Slightly eclipsed by ASAN these days, but it’s still vitally necessary to test your release builds under valgrind because it will catch things that ASAN builds can’t.