The “using” does involve judgement, and not based on weighting X and Y to create a new combined metric. It has much more actionable guidance than just “use your judgement” though.
I find this interesting to contrast with another entry on the front-page today, Ratchets in software development: The other article provides what seems like a counter-example to the strong version of Goodhart’s Law: The count of locally-deprecated uses of an API is successfully used as both a measure and a target. To be a bit more explicit:
A value is to encourage common understanding and use of current engineering patterns;
The measure is the count of how many patterns contrary to what local engineers currently prefer exist in the code-base;
This is targeted such that deprecated patterns monotonically decrease across the lifetime of the code-base.
100% honest use of a metric, taken far enough, is probably contrary to the orgs goals (acknowledged in the other article, as the author allows that circumstances can allow the target to increase sometimes)… But the “far enough” clause is doing a lot of work. In this case at least, for a fair-minded reader, Goodhart’s Law doesn’t seem to have held.
Edit to add: I think the important variable in whether Goodhart’s Law exhibits or not will be how aligned incentives are. If the folks who use the measure have the same incentives as those who affect the measure, then Goodhart’s Law will not apply. OTOH, if these incentives go out of alignment (e.g. a PHB observes “the number of deprecated uses hasn’t gone down in 6 months” with some form of implicit or explicit threat), then expect the negative implications of the law to arise.
That article is interesting in conjunction with ~lcapaldo’s comment on using metrics… That ratchet mechanism is a very dumb metric, and a good part of that article is taken up by explaining exactly how dumb it is. But that’s fine because the goal of the metric isn’t to reduce use of that sort of code to zero. That would have costs that are basically unrelated to improving the operation of the program. The goal is to not let it increase, which is much cheaper (in terms of code not needing to be rewritten/modified/re-tested/etc) and still a good proxy for the real goal, which is something like “make future development easier, in this case by avoiding particular antipattern-prone constructs”.
I think there’s a nice parallel between systems falling foul of Goodhart’s Law and AI systems erroneously overfitting their training data.
It’s said that when optimising for a particular parameter in the extreme, all unconstrained parameters invariably assume extreme values. Optimising for paperclips? You’ll end up with human extinction and grey goo-ification.
There’s a comment at the end of this post mentioning that you could combine multiple metrics into one, but this is rapidly brushed off: “but that too is a metric subject to Goodhart”. I think this is throwing the baby out with the bathwater. While, yes, this is true in the extreme, composite metrics are more likely to avoid extreme unconstrained parameters because they’re an attempt to constraint parameters that would otherwise be unconstrained.
When you’re trying to optimise for many competing but quantifiable goals that are chosen to each be somewhat aligned with an unquantifiable goal, you’re more likely to end up accidentally optimising for the unquantifiable goal too.
I think a good alternative - and one that’s recently become popular in AI safety too - is that of satisficing. Set yourself a goal, but give it a threshold. Work to reach that threshold, but don’t make any attempt to push beyond it. In doing so, you’re more likely to avoid overfitting and hence also more likely to align your actions with your underlying unquantifiable goal.
I found this to be at least persuasive guidance how to “use a combination of metrics”: Goodhart’s Law Isn’t as Useful as You Might Think.
The “using” does involve judgement, and not based on weighting X and Y to create a new combined metric. It has much more actionable guidance than just “use your judgement” though.
I find this interesting to contrast with another entry on the front-page today, Ratchets in software development: The other article provides what seems like a counter-example to the strong version of Goodhart’s Law: The count of locally-deprecated uses of an API is successfully used as both a measure and a target. To be a bit more explicit:
100% honest use of a metric, taken far enough, is probably contrary to the orgs goals (acknowledged in the other article, as the author allows that circumstances can allow the target to increase sometimes)… But the “far enough” clause is doing a lot of work. In this case at least, for a fair-minded reader, Goodhart’s Law doesn’t seem to have held.
Edit to add: I think the important variable in whether Goodhart’s Law exhibits or not will be how aligned incentives are. If the folks who use the measure have the same incentives as those who affect the measure, then Goodhart’s Law will not apply. OTOH, if these incentives go out of alignment (e.g. a PHB observes “the number of deprecated uses hasn’t gone down in 6 months” with some form of implicit or explicit threat), then expect the negative implications of the law to arise.
That article is interesting in conjunction with ~lcapaldo’s comment on using metrics… That ratchet mechanism is a very dumb metric, and a good part of that article is taken up by explaining exactly how dumb it is. But that’s fine because the goal of the metric isn’t to reduce use of that sort of code to zero. That would have costs that are basically unrelated to improving the operation of the program. The goal is to not let it increase, which is much cheaper (in terms of code not needing to be rewritten/modified/re-tested/etc) and still a good proxy for the real goal, which is something like “make future development easier, in this case by avoiding particular antipattern-prone constructs”.
I think there’s a nice parallel between systems falling foul of Goodhart’s Law and AI systems erroneously overfitting their training data.
It’s said that when optimising for a particular parameter in the extreme, all unconstrained parameters invariably assume extreme values. Optimising for paperclips? You’ll end up with human extinction and grey goo-ification.
There’s a comment at the end of this post mentioning that you could combine multiple metrics into one, but this is rapidly brushed off: “but that too is a metric subject to Goodhart”. I think this is throwing the baby out with the bathwater. While, yes, this is true in the extreme, composite metrics are more likely to avoid extreme unconstrained parameters because they’re an attempt to constraint parameters that would otherwise be unconstrained.
When you’re trying to optimise for many competing but quantifiable goals that are chosen to each be somewhat aligned with an unquantifiable goal, you’re more likely to end up accidentally optimising for the unquantifiable goal too.
I think a good alternative - and one that’s recently become popular in AI safety too - is that of satisficing. Set yourself a goal, but give it a threshold. Work to reach that threshold, but don’t make any attempt to push beyond it. In doing so, you’re more likely to avoid overfitting and hence also more likely to align your actions with your underlying unquantifiable goal.