Neat stuff. Is that how the deadlock detection in e.g. Go works?
It looks very similar to the WITNESS subsystem in FreeBSD. If you’ve ever run a non-release kernel and see ‘Lock order reversal in…’ messages show up in dmesg or on the system console, this is where they come from: the system tracks which mutexes are held when another is acquired and then dumps a stack trace any time they are acquired in the reverse order (e.g. if the first acquisition of B has A locked and later code tries to acquire A while holding B locked then you’ll get a report).
There are a few situations in which this can provide false positives. In particular, a couple of things have (for legacy reasons that are avoided in new code) different lock order rules during initialisation and normal operation. In these cases, the first lock order is A then B, the later one is B then A, but all code that acquires them in the A then B order finishes running before the code that acquires them in the B then A order. In general, it’s easier to ‘fix’ these cases than try to validate that the inversion is safe.
There was a GSoC project a long time ago to port WITNESS to the userspace pthreads implementation. I don’t know what happened to it, I don’t think it was ever merged.