Here’s my take on it:

test "example" {
    foo();
}

fn foo() i32 {
    return 1234;
}

./test.zig:2:8: error: expression value is ignored
    foo();
       ^

Aha.

My Hobby Horse.

Let me ride it.

I have fixed over the decades a vast number of bugs by the “One Simple Trick”.

Grepping for every place where my predecessor had cast a return code to void (or switching on the gcc warnings that tell me where return codes are being ignored).

And then turning every darn one into a YELL and crash now.

Ooo. Looky you guys have been hiding creepy crawlies all over the place, let’s just fix them!

See, now the software is rock solid instead of flaky sometimes doesn’t work for no reason.

Now let’s add some subtlety.

In the case the values are coming from an untrusted external system. aka The User. The config should have passed a stringent validation check before replacing the old.

Sounds like they didn’t even have a validation check.

You should be able to fuzz the hell out the validation checker and every config that passes should load successfully.

Ah, but you’ve wired the “load config” to the “do stuff” haven’t you? Bad Idea. Config is Plain Old Data without behaviour, you should be able to round trip it without involving the rest of the system.

Ok, if you get passed all that, yes you have a bug. The sooner you find it and fix it, the less shit downstream you have. In this case the result was dramatic and obvious… all died.

A far far far worse scenario is it limped on with an undefined random value that resulted in corrupted data / transactions over months requiring manual check and correction of 1000’s of them.

Better to die on start up, and if it dies, rollback to the previous config.

Next is if system A dying kills system B, you’re lying if you think they’re independent standalone systems. Systems die, people trip over the power cord, diggers dig up cables, operators have carrots for fingers and potatos for brains… What is your recovery strategy to recover the whole system?

Handling error returns? Careful there, dragons lie that way, complex code, resource leaks, more and more bugs….. and in the end? Do you know how to handle it? Or are you just passing it up and up and up hoping someone knows what to do…. and the only thing they do is drop it on the floor and hide the bug.

Rather be honest, if you haven’t a tested and working plan to handle it? Crash Now and fix it!

In an in production system a crash now and restart usually results in less loss of service than degrading into an flaky, untested, undefined state that no one knows about.

What I miss in this article is that ‘it makes the code messy’ is actually a very good argument. The reply shouldn’t have been ‘you should have done it anyway’, but ‘why didn’t you invest in a way to check for errors without making the code messy?’

And if the answer was ‘there was no budget for that project’, then you know the real source of the problem.

1. [[nodiscard]] is one of the greatest things to ever happen in C++.
2. Assume any pointer (or equivalent) you get from something could be null unless proven otherwise.

The description of the bug here is a bit too vague for my taste. It’s not really clear that this is a simple “ignore checking for return value”, it sounds more like they weren’t doing in-depth validation of some complex value.

E.g., this could have been

getConfig(params) {
  if (condition) {
    return { mode: 1, b: bar }
  } else {
    return { mode: 2, c: baz }
  }
}

and then code later on crashed because it was expecting cfg.b to be defined. Now it seems not that clear to require to “validate the config to have the expected fields everywhere you fetch it”; the value returned by getConfig is always non-zero, and you should be able to rely on it satisfying whatever contract is agreed on. I’m sure there would have been mistakes made here, too, but it might just be a little bit more subtle than merely not checking for a NULL function return value.

Anyway, I don’t doubt that there was a real cultural issue here, I’d just think the problem has to be described a little more concretely to actually pinpoint that.

My colleagues thought I was wasting my time writing validation for configuration; known sane defaults, alerting when required values are missing and ensuring strict type checks.

I still get tripped up on trusting sources when by now, I should know better. The last time was a few days ago when I learned (the hard way) that client certificates don’t have to have a subject field (completely optional! Who knew?). The previous was trusting the Oligarchic Cell Phone Companies to send valid phone numbers (thank God they’re on trusted networks, right? Right? Where did everybody go?). The time before that was when a sentinel “don’t delete this value from the database” was deleted from the database (which caused a cascade failure of our service—what fun).

Yeah, check everything at the I/O boundary.

Feels like half this conversation isn’t talking about what the problem is actually. It’s probably the equivalent of this:

# config.json
{
 "number_of_threads": 1
}

#load_config.py
def get_num_threads():
  config = json.loads("/my/config.json")
  return int(config.get('num_threads'))

This is totally an API thing. Scala Map[A,B]#get takes an A and returns an Option[B]. So does Python, but you’ll only discover it at runtime if you aren’t familiar with the API. Java returns a B which in the world of Java means B | null. Python’s get returns B | None but B is less constrained. So yeah, when you use a tool you’ve got to know all the pieces of the tool. Tool which needs less memorization of its idiosyncrasies is better than other tool. Lets you operate at higher level.

While it might be verbose, Go’s tooling/idioms were designed to stop this thought process.

returnval, err := doThing()
if err != nil {
    return err
}

err has to be used to avoid a compiler error. And the error handling/return value checking is idiomatic to the point that it would look wrong without it. Furthermore, failing fast on error leads to minimized indentation (so it looks prettier by the Go aesthetic).

Author carefully tries not to point fingers at the language, but “you should check all error conditions | but I don’t want to!” is a very old problem, and different languages have tried to solve it in various ways. I think we’ve had a meaningful progress since ALGOL 68 in this area:

• Making handling less messy with less burdensome syntax for error propagation (like ? in Rust)
• Propagating missing values instead of blowing up (like ?. in Kotlin/TypeScript)
• Making checks mandatory with optional types. If the program blows up, it’s because you wrote force-unwrapping call that makes it so.
• Removing null entirely. Things can only be missing if they’ve been explicitly designed to be optional. That makes “but it was supposed to be there!” less of an excuse.

I haven’t seen a solution that fixes the problem entirely, but in my experience these features can reduce it from a common problem that can happen by accident, to exceptional cases which need either pretty complex data flow and/or code that is cutting corners in ways that are easy to notice in code reviews.

That sort of attitude from developers is super depressing. I think there are multiple possible solutions, but the culture that leads to ignoring the problem is one that needs to be fixed. Really there’s two responses to finding an issue like this:

1. Paper it over so that crash doesn’t happen (hey, sometimes this is the best you can do)
2. Go one layer deeper and try to eliminate a whole class of similar bugs

The problem with #1 is you get to deal with the same bug over and over.

There are a few ways to do #2. You can use a language without the “billion dollar mistake”. For other languages you can at least validate a config at load time centrally before returning it for use which, from the author’s description, seems like the right thing to do here.

There are two good approaches to that issue.

The first one is to check and handle all possible unexpected results, and have tooling to ensure you do it.

The second one is “let it crash”, famous in Erlang circles. Basically, when an error occurs, log it so you can debug it and panic the process and let a process manager restart it in a known state (“process” does not necessarily mean “OS process” here).

In general, you would use both: very critical code (think DB transactions, etc) constitutes an “error kernel” which you write with the first approach, and all business code around that uses the second one.

Idiomatic Elixir example:

{:ok, file} = File.open("hello", [:write])

Enforced checking by returning a tuple of {:ok, the-return-value} on success and {:error, something-else} otherwise.

This can get ridiculous, though, so some functions are named with a ‘!’ suffix and just return the bare return value - but throw if there’s a problem.

It’s about line of defense. The article drives home the point of defensive programming, but fails to talk about exactly where defensive programming is appropriate.

You have to draw a line in the sand: On one side, you have untrusted input, where defensive programming is appropriate. The line itself is input validation, which turns untrusted input into errors and trusted input. Beyond this input validation, it is arguably offensive programming, aka. let it fail, that is appropriate – any failure here indicates breach of contract.

The programmer said: “It’s not supposed to be like that, so we don’t need the checks.”

Correct – given that we are inside the line of defense! I don’t support the author’s automatic dismissal of this argument. The right question is where, along the whole information chain from the user to where the crash happened, was input validation supposed to happen.

This is definitely one of those places where web dev has pushed the state of the art forward. I think in particular stuff like Django’s template library having the notion of “safe strings” (and other things always getting escaped).

Like seriously, check your inputs! And your input should be a different shape from the result output (to avoid accidental usage).

Rust helps with the ergonomics of this with Result + ?, and depending on your context you can make it nice.

The way[1] to “force” return value checks is to instead provide a result on success, and an error on failure, so there’s no way to get the “thing you want” without checking code.

[1]: not always applicable but…. applicable in so many places

Good post and discussion.

This doesn’t handle memory outside the system. This doesn’t handle IO, network, database etc. You cannot avoid all runtime errors so there are no guarantees. You cannot invent a language or a machine/cpu/fpga that doesn’t not have some kind of runtime errors. Yes, all of us with our different machines and languages all need to do error handling, testing and monitoring and much more than that. Types and compilers are helpful but many people think that there is some kind of magic and then are surprised when it fails. This also doesn’t mean “don’t use types and compilers”. You need a mix of tools but also you need to understand what will never be possible and why there is no silver bullet.

• If I add typescript I don’t need tests. No.
• My compiler will catch all my mistakes. No.
• The new version of my language/framework will fix this. No.
• The next trendy language will have no errors of any kind. No.

What works (probably) but we (whoever that is, definitely includes me) don’t do enough of is:

• Feedback on pain points (money/time)
• Add tooling
• Take time to teach to open minds (not a time issue)
• Change culture (but how and with what authority)

Maybe some approaches for the original story:

Add a test to simulate the country that is having problems. It fails. Make the test pass. Decide if there is a refactoring opportunity. Refactor (or not). Commit the tests and the fix. The repo is stronger. There’s a monitoring angle here too but to me monitoring is a different kind of testing - it’s the same test just continuous.

If the problem is all I/O, then you can’t control the memory so all your types and compilers have no power. You can let it crash but it still doesn’t work. If you want to avoid or know breakage there is contract testing which is pretty neat in this era of mashups and cloud. See pact.io, it’s weirder but easier than it seems. You need to believe pact a bit though, it’s slightly different than maybe what you’ve seen. It’s a different layer of tests. Ultimately, this is probably culture but I like introducing tools as enforcement/codification for cultural values.

Obviously, test early, test often. Everything that can fail, should have a pre- and post-condition check. Yes, that means:

const result = somecall(x);
if ( ! resultValid(result)) {
    throw new Error(...);
}

And somecall should validate all its parameters the same way. Ideally you’d trust somecall to validate its result, but in reality it’s some library you don’t own and can’t trust.

Often in production code, you go along thinking it can’t fail, and then it does, and then you instrument the entire section with more tests.

I haven’t heard anyone say “it’s messy” before, but “we’re time-constrained”, or “we didn’t think it could fail” are very common. And there’s a subset of soi-disant programmers who will not write tests, no matter what you do, and you either code around them, treat them like faulty hardware, or fire them.