Sorry, but it’s silly to call something a “unit test” which “only dependencies being the operating system”.
I define a unit test for functions only where the same input always provides the same output with no side effects (“pure” functions). If you need to mock something (file, os, I/O, whatever), or a test that has side effects, that’s an integration test (it integrates with the system/storage/network/whatever).
Why is this important? The naming is not that important, but if you think about this way, maybe you can realize how important is it to split code and abstractions on I/O boundaries, so you can test main logic without ever touching disk or network.
I’ve never seen this definition of “unit test”, and it seems unnecessarily restrictive. What about a very simple class that holds a single state variable which is manipulated through a getter and setter? Most people would definitely call it a unit test when you’re verifying that the getter can read what the setter set.
Also, AFAIK all web frameworks call ORM model tests “unit tests”, even though they go through a database. It’s about the smallest unit you can meaningfully test in a web framework.
I define a unit test for functions only where the same input always provides the same output with no side effects (“pure” functions). If you need to mock something (file, os, I/O, whatever), or a test that has side effects, that’s an integration test (it integrates with the system/storage/network/whatever).
That’s probably approximately what “unit test” means in common parlance today, but that’s not the original definition of the term, as far as I know. To the best of my knowledge, a “unit” originally was just a “unit of functionality” in a piece of software. So, if you had a piece of software that had an “export to CSV” button, a unit test might provide some hand-made data to the export_to_csv function and see that it actually writes a valid csv file to disk with the data represented correctly.
Not that I’m arguing for one definition over the other, just that I think your opening statement of calling one definition “silly” is probably misguided, especially considering that I think you’re calling the original definition silly.
Nothing authoritative, unfortunately. It’s just something I’ve picked up over the years, which is why I used such uncertain language in my comment (“To the best of my knowledge”, etc). It’s mostly just something I picked up from people who had been programming much longer than me.
However, I do have a fairly old post from Martin Fowler in my bookmarks that discusses object mocking in the context of Test Driven Development and the two “schools” of TDD: London and Classical. The post doesn’t directly set out to define “unit test”, but his explanations and examples kind of hint at the idea that a unit test isn’t necessarily a single-class or single-function affair, and the post is from 2007: https://martinfowler.com/articles/mocksArentStubs.html. See, for example, the section titled “Classical and Mockist Testing.”
I find this distinction a bit confusing since there’s lots of cross cutting concerns which may be io boundaries, but also seem pure. For example anything that includes logging. In an extreme case add(a,b,logger) { logger.log("adding"); a+b } - I’d argue a test for this is not an integration test, but it may be just one frame away from invoking io and requires effectively mocking the logger.
Amusing, but in practice - this is dogma taken too far, right? This function didn’t need this level of unit testing, and this is why programming teams set things like 70% unit test coverage goals. This function is part of a larger project, and some of the other parts will need more unit testing that this one, so only achieving 50% on this function is acceptable.
The team needs to set and enforce standards, but they don’t want to create something that becomes useless dogma, so they say 70%.
But that doesn’t make unit testing useless. It can still catch obvious things, clarify intended behavior, highlight regression… Etc.
Perhaps this guy’s manager at The Enterprise was just commonly an enforcer of meaningless dogma.
You’re probably right about my former manager (I’m no longer there), he was all about the tests. It got to me when he asked for negative tests (test that something didn’t happen).
So how much testing should that function have? I know that error paths are notorious for not being fully tested, which is pretty much half the function.
Those error paths feel like better fits for something closer to integration testing. I tend to think of a continuum from integration testing to unit testing with many gray points in between, so “unit testing” of a slightly larger chunk of the system to include your function here doesn’t seem impossible or unwise to me necessarily. Although it goes against some of the definitions here.
That would highlight how hook failure and system failure interact with your code and whether the function is even implementing the expected handling for the rest of the system, at the same time as whether the failure handling is working.
The error handling lines themselves are so simple to reason about. I would find it really frustrating if someone pushed me to write unit testing for that just to meet some metric. (Something I have to remember to avoid as a manager.)
Sorry, but it’s silly to call something a “unit test” which “only dependencies being the operating system”.
I define a unit test for functions only where the same input always provides the same output with no side effects (“pure” functions). If you need to mock something (file, os, I/O, whatever), or a test that has side effects, that’s an integration test (it integrates with the system/storage/network/whatever).
Why is this important? The naming is not that important, but if you think about this way, maybe you can realize how important is it to split code and abstractions on I/O boundaries, so you can test main logic without ever touching disk or network.
I’ve never seen this definition of “unit test”, and it seems unnecessarily restrictive. What about a very simple class that holds a single state variable which is manipulated through a getter and setter? Most people would definitely call it a unit test when you’re verifying that the getter can read what the setter set.
Also, AFAIK all web frameworks call ORM model tests “unit tests”, even though they go through a database. It’s about the smallest unit you can meaningfully test in a web framework.
That’s probably approximately what “unit test” means in common parlance today, but that’s not the original definition of the term, as far as I know. To the best of my knowledge, a “unit” originally was just a “unit of functionality” in a piece of software. So, if you had a piece of software that had an “export to CSV” button, a unit test might provide some hand-made data to the export_to_csv function and see that it actually writes a valid csv file to disk with the data represented correctly.
Not that I’m arguing for one definition over the other, just that I think your opening statement of calling one definition “silly” is probably misguided, especially considering that I think you’re calling the original definition silly.
Do you have any links for the original definition? Couple of days ago I looked at the docs for JUnit 1.0, but they didn’t define the unit.
Nothing authoritative, unfortunately. It’s just something I’ve picked up over the years, which is why I used such uncertain language in my comment (“To the best of my knowledge”, etc). It’s mostly just something I picked up from people who had been programming much longer than me.
However, I do have a fairly old post from Martin Fowler in my bookmarks that discusses object mocking in the context of Test Driven Development and the two “schools” of TDD: London and Classical. The post doesn’t directly set out to define “unit test”, but his explanations and examples kind of hint at the idea that a unit test isn’t necessarily a single-class or single-function affair, and the post is from 2007: https://martinfowler.com/articles/mocksArentStubs.html. See, for example, the section titled “Classical and Mockist Testing.”
I find this distinction a bit confusing since there’s lots of cross cutting concerns which may be io boundaries, but also seem pure. For example anything that includes logging. In an extreme case
add(a,b,logger) { logger.log("adding"); a+b }- I’d argue a test for this is not an integration test, but it may be just one frame away from invoking io and requires effectively mocking the logger.Amusing, but in practice - this is dogma taken too far, right? This function didn’t need this level of unit testing, and this is why programming teams set things like 70% unit test coverage goals. This function is part of a larger project, and some of the other parts will need more unit testing that this one, so only achieving 50% on this function is acceptable.
The team needs to set and enforce standards, but they don’t want to create something that becomes useless dogma, so they say 70%.
But that doesn’t make unit testing useless. It can still catch obvious things, clarify intended behavior, highlight regression… Etc.
Perhaps this guy’s manager at The Enterprise was just commonly an enforcer of meaningless dogma.
You’re probably right about my former manager (I’m no longer there), he was all about the tests. It got to me when he asked for negative tests (test that something didn’t happen).
So how much testing should that function have? I know that error paths are notorious for not being fully tested, which is pretty much half the function.
I honestly don’t know.
Those error paths feel like better fits for something closer to integration testing. I tend to think of a continuum from integration testing to unit testing with many gray points in between, so “unit testing” of a slightly larger chunk of the system to include your function here doesn’t seem impossible or unwise to me necessarily. Although it goes against some of the definitions here.
That would highlight how hook failure and system failure interact with your code and whether the function is even implementing the expected handling for the rest of the system, at the same time as whether the failure handling is working.
The error handling lines themselves are so simple to reason about. I would find it really frustrating if someone pushed me to write unit testing for that just to meet some metric. (Something I have to remember to avoid as a manager.)