This can get worse than that. If the work take more than the length of the interval, you can run into problem where a dumb scheduler can start another instance of the process while the other is still running. This can lead to all sort of nasty problems:
the two processes can trash each others if they share resources, which is likely, since they are two instance of the same program. (Though this is relatively easy to fix using locking or a similar mechanism)
They can start banging an external system with twice as many request. This is especially nasty, because if it slow down the system enough, it can lead to a third process getting spawned, compounding the problem.
I’ll also note than depending on the nature of the problem, you cannot always use the “wait from the time it end” solution: At a previous employer, we had a case where the “nightly” billing job that generated the invoices was getting dangerously close to taking 24h, meaning that it would start overlapping the next billing job. Since invoice have to be generated on a given date, this was quite problematic. Thankfully, the problem was embarrassingly parallel, and we fixed by partitioning the problem and running it on multiple machine.
Feel free to burn your own resources. Where it becomes troublesome is when it reaches out and starts burning those of other people.
Hang on, running CPU cores at full speed with no output because a busy loop means you don’t need to think your automation through any further, isn’t that pretty much a canonical example of burning other people’s energy resources? Isn’t it a classic example of why the world is where it is right now?
In this example I would have retry logic (with back off etc.) in my program, otherwise it relies to much on me knowing when I last hit some service or the other.
This can get worse than that. If the work take more than the length of the interval, you can run into problem where a dumb scheduler can start another instance of the process while the other is still running. This can lead to all sort of nasty problems:
I’ll also note than depending on the nature of the problem, you cannot always use the “wait from the time it end” solution: At a previous employer, we had a case where the “nightly” billing job that generated the invoices was getting dangerously close to taking 24h, meaning that it would start overlapping the next billing job. Since invoice have to be generated on a given date, this was quite problematic. Thankfully, the problem was embarrassingly parallel, and we fixed by partitioning the problem and running it on multiple machine.
i wrap most of my cron commands with flock, for this reason
Hang on, running CPU cores at full speed with no output because a busy loop means you don’t need to think your automation through any further, isn’t that pretty much a canonical example of burning other people’s energy resources? Isn’t it a classic example of why the world is where it is right now?
This post was a nice reminder, it’s very easy to forget subtle issues like this!
In this example I would have retry logic (with back off etc.) in my program, otherwise it relies to much on me knowing when I last hit some service or the other.