The thing is, figuring out all the exceptions is not free. And often the exceptional cases are rare enough that the benefit is minimal.
unit tests aren’t very helpful in testing a cryptographically secure random number generator.
Bollocks. That’s a perfect use case for them.
unless you can write a unit test to determine awesomeness.
Is that supposed to be hard?
What would a “unit test” for a cryptographically secure random number generator look like? IIRC you run a battery of statistical tests against A LOT of data from the RNG to get a probability that the RNG is good e.g. https://en.wikipedia.org/wiki/Diehard_tests
In my eyes the point of unit testing is a low effort method to catch obvious failures. If you’re only looking for obvious failures in an RNG you don’t need to do huge amounts of work.
ITHare has a good article on RNG, and briefly discusses testing.
I see it as not a good use case for them as the length of time you’d have to run it to actually get red/green results is astronomical. What makes you feel so strongly it’s a good case for them?
Don’t forget these are typically pseudo-random number generators.
ie. Given a particular seed, you always get exactly the same sequence of bits out of them.
Suddenly that makes it really obvious how you could do some tests…
Also there are typically several substeps and data structures involved, all of which are very amenable to unit testing.
Or let’s consider the traditional “hard” variety.
A hardware random number generator.
No unit tests for that eh?
Here is reference for one of them….
Quite a bit of code in there.
How do you unit test it?
Well, you create a shim that allows you mock the machine code instructions that actually touch the hardware.
And then you control the mock to return every interesting value, or status return and verify your code can handle it.
Often your driver has to enable certain lines, and initialize and configure the hardware. Does it?
Often it has to load / store certain memory addresses, or use specific instructions? Does it?
Sometimes it is suppose reseed after every N pseudo random bits…. Does it?
Well that’s not true at all. There are a ton of statistical tests for judging random number generators, none of which require an “astronomical” amount of time.
I personally don’t think that kind of test belongs in a unit test suite, though.
Here’s a long essay on testing RNGs. [PDF]
I appreciate the intent here, but that’s not how people learn new practices/skills. An expert became an expert not by thinking really hard, but by doing things that worked in their context, and continued to apply it until it stopped working (which isn’t always obvious).
If you’re an expert, make sure you explain the qualifications to your statements.
Alas, the novices won’t always understand the nuance and look to straightforward prescriptive advice that they can blindly follow, because that’s how many people learn (this sums up the current state of Agile, to me). At some point, the novices advance sufficiently so they can ask the boundary questions and understand them.
You need to take every piece of advice and figure out when and where it does not apply.
But also understand that where the “expert” believes it applies (or doesn’t) is based on their personal experience, and yours may (and will) differ. Just because it didn’t work for the “expert” in a certain situation, doesn’t mean it won’t work for you (and vice-versa).
There’s also a matter of discipline: doing things like TDD or test-first is often harder than writing tests later (if at all), so having a rule of “I’ll always write tests first for production code” is much easier to follow (especially when peers reinforce this) than saying “in these situations I’ll write tests first”.
The code/test cycle is always somewhat iterative, and I think that’s fine. I’m sure no one ever got fired for drafting some code, writing a test, making the code better and adding more tests.
The thing I consider here is that saying “Write tests” is a sloppy way of putting it which invites sloppiness, so adhering to a religion while committing the occasional sin is actually fine.
As long as you don’t commit that sin to Git.
TDD (and Agile) is fundamentally about delivering high quality at the highest speed possible. Not writing tests immediately will give more immediate speed but no guarantee of quality.
Writing tests after the fact, much later, is a terrible task.
So it’s ok to preach TDD, and pay attention only to whether or not the push comes with tests or not.
For the purpose of Git bisect, which is a real use case, having a commit that fails because of missing code is not useful. Maybe that didn’t exist 15 years ago, but it proves that “firstness” and code commits are a blurry line.
For the purpose of Git bisect, which is a real use case, having a commit that fails because of missing code is not useful.
It’s not a problem either though; it’s easy enough to skip on compilation errors.