I’ve tried to do similarly. I didn’t put data into code files, because I didn’t see that as useful, but I did try to substitute a ‘file system’ for the real one when running tests.
My ‘file system’ was simple in concept: Pick a directory in the real filesystem and pass its path to the test code. If we are only doing read-only testing, all we then have to do is pretend this directory is the root of the space used by the code.
I’ve also do this where - because I’ll be manipulating files - I copy the whole set of data into a temporary directory and use its path as root, destroying after each test, of course.
I didn’t quite understand all the assertions made in the article.
Yes, unit tests should run fast. I’ve never needed so much data for unit tests that it’s ever caused me to notice an impact on speed. I try to keep tests to the minimum of inputs / variables, so that it’s clear what’s being tested. This goes the same for (unit) test data in files. I only use big files for performance tests.
The thing about running cmake explicitly … seems to be a bit of post-justification for writing a fun bit this code.
Tests should be repeatable, yes, but then there is an assertion that they shouldn’t share data … and they shouldn’t do so through files.
Using files doesn’t mean that tests share data. Putting that data in code files doesn’t mean it’s somehow immune from abuse in the test code. Just make your setup and tear down enforce that you have a clean slate each time.
My main reason for moving to using real files (I too started off thinking I should put data in memory) was that I wanted to get the actual behaviour of the filesystem. The right errors on missing files, wrong permissions, etc. I actually wanted to deal with permissions issues by informing the user exactly what seemed to be up, so getting the correct error was vital.
The problem I had with writing the code is that there wasn’t a way (that I could figure out) of mocking (sorry, making a test double?) the filesystem without having to write a whole abstraction layer for the real code to use - and to port it to that. It wasn’t much work, but being able to swap out a filesystem should be easy. I think if I’d been writing my code for UNIX style systems I could have more easily (perhaps just to more experience on my part) done some tricks with mounting and chrooting, if this was fast enough.
I’m not sure what separating tests from the filesystem accomplishes. Tests that use the filesystem are fast and deterministic. Adding a double here is work for not much benefit.
I’ve tried to do similarly. I didn’t put data into code files, because I didn’t see that as useful, but I did try to substitute a ‘file system’ for the real one when running tests.
My ‘file system’ was simple in concept: Pick a directory in the real filesystem and pass its path to the test code. If we are only doing read-only testing, all we then have to do is pretend this directory is the root of the space used by the code.
I’ve also do this where - because I’ll be manipulating files - I copy the whole set of data into a temporary directory and use its path as root, destroying after each test, of course.
I didn’t quite understand all the assertions made in the article.
Yes, unit tests should run fast. I’ve never needed so much data for unit tests that it’s ever caused me to notice an impact on speed. I try to keep tests to the minimum of inputs / variables, so that it’s clear what’s being tested. This goes the same for (unit) test data in files. I only use big files for performance tests.
The thing about running cmake explicitly … seems to be a bit of post-justification for writing a fun bit this code.
Tests should be repeatable, yes, but then there is an assertion that they shouldn’t share data … and they shouldn’t do so through files.
Using files doesn’t mean that tests share data. Putting that data in code files doesn’t mean it’s somehow immune from abuse in the test code. Just make your setup and tear down enforce that you have a clean slate each time.
My main reason for moving to using real files (I too started off thinking I should put data in memory) was that I wanted to get the actual behaviour of the filesystem. The right errors on missing files, wrong permissions, etc. I actually wanted to deal with permissions issues by informing the user exactly what seemed to be up, so getting the correct error was vital.
The problem I had with writing the code is that there wasn’t a way (that I could figure out) of mocking (sorry, making a test double?) the filesystem without having to write a whole abstraction layer for the real code to use - and to port it to that. It wasn’t much work, but being able to swap out a filesystem should be easy. I think if I’d been writing my code for UNIX style systems I could have more easily (perhaps just to more experience on my part) done some tricks with mounting and chrooting, if this was fast enough.
I’m not sure what separating tests from the filesystem accomplishes. Tests that use the filesystem are fast and deterministic. Adding a double here is work for not much benefit.