Interesting. Would be even better with some instructions. This seems like a legitimate and useful use of such models. Integrated with an editor to lint unusual choices this could definitely be quite supportive.
Combined with any machine comprehensible way of checking correctness this might be even more powerful.
This reminds me of how you can have a chess engine analyze your game after the fact to go “this was a good move”, “this was a mistake”, and so on.
Of course, the big difference is that the chess engine is guaranteed 100% to be better than you, which is not (yet) true for AI writing software.