The only thing I would add to that post is that static types are not required to gracefully avoid nulls. Erlang by convention returns values from operations that may fail wrapped in a tuple like {ok, Value} which forces you to pattern-match in order to unwrap the tuple. This indicates that you need another clause in your pattern match to catch non-ok values. (Though of course in Erlang it’s permissable to fail to handle the error case and just let your process supervisor tree respawn the process.)
There’s nothing stopping communities from adopting safer conventions in languages like C or Java.
The thing that makes Erlang “better” is that the conventions have already been adopted broadly, where as in C, or Java there’s still a pretty long, very steep hill to climb.
Java added an Option type in SE 8, and, of course Guava had one prior to that – a convention worth adopting for sure.
Java added an Option type in SE 8, and, of course Guava had one prior to that – a convention worth adopting for sure.
But you still need null checks (or have appropriate annotations I guess) in every method that accepts Options, since it is just another object and can be null.
Or you can adopt a convention that you don’t pass nulls, and check only at the boundaries or when interfacing with libraries that haven’t adopted that convention. Which puts you in the same place as Erlang.
It’s not the same place as Erlang; the conventions have been part of Erlang’s standard library for decades, and Erlang’s syntax makes it basically effortless to deal with their Option types. Java’s is brand new, and the fact that it’s still nullable means it’s really hard to convince people to use it because it doesn’t offer useful guarantees, so it’s no surprise it’s seen very little uptake.
That’s true. It’s putting lipstick on a pig, or something. At some point programmers need to take some responsibility. If your method returns an Option, maybe you should write enough tests to ensure it won’t return null.
While true erlang’s functional nature and runtime makes it a little less problematic to have your processes die all over the place as well. It’s not just the conventions they built up it’s also the way they built their runtime around the expectation that NPE’s would happen basically all the time.
For sure beam is designed to handle failures gracefully, but are these failures really equivalent to NPEs? Or is there just a tendency to handle errors by exiting the process?
I don’t know Erlang very well, but my sense was always that the “let it crash” philosophy was more about the uncertainty of running distributed systems–can’t connect/lose the connection? “Let it crash!” Instead of, “Go ahead! Be a sloppy programmer!”
You’re correct; it’s very different. An NPE in Java means “some programmer screwed up” while an Erlang crash usually means “some external situation prevented us from proceeding, so we should try again”.
Both can cover both meanings though. A java program can deliberately allow a RuntimeException to propagate for a transient failure knowing a retry will happen at high-level - I’ve worked on an application that was structured like this (basically a pipeline of queues, if processing a given message failed we’d retry it a few times and then error it out). And an Erlang crash can be a programmer screwup.
Do you mean that “Not Applicable”, or that the answer is “Not Found” in your brain, your brain is “Not Initialized”, or your brain is in an error state, or you haven’t thought about it yet? :-D
Funny, it’s almost as if representing a number of disparate failure conditions using a single value results in a lot of ambiguity and easily leads to misunderstanding. Who would have thought!
Null makes sense in a type system that lacks sum types and a way to require pointers be initialized.
For example, the closest thing to sum type column in SQL is to have multiple tables with foreign keys to the record you’d want to have the column on, plus a constraint. Rather than every blog_post having a nullable published_atdatetime, have a table publishing with an FK pointing at blog_post, an atdatetime, and a constraint specifying exactly 0 or 1 of them exist. That’s a lot of hoops to jump through (extra records, thinking through a join instead of just saying published_at is null, etc.) and implies something that might not be part of your domain (a blog_post could be published multiple times, at first glance) to represent something that’s adequately expressed by a nullable type.
Or in C++, when the language doesn’t require that int * foo get initialized to a valid int before it can be passed around and dereferenced. There’s something in there, right? Having a concept for null strikes me as marginally less bad than passing garbage values around without a name, but I dunno.
I’d put it on the big pile of programming concepts that seemed like a decent idea at the time but are really problems better solved differently.
(Also for context, this topic came out of chat about this story.)
No. Null/zero values are a useful, safer alternative to uninitialized values. The mistakes are 1) bad type systems that don’t properly account for null 2) programming with raw memory structures directly most of the time and 3) pervasive library code that don’t do sane things with zero values.
What would you say is the right way to deal with null? Force nullable pointer users to branch on whether the pointer is null? That seems just like options, only without the convenience of options being opt-in. As far as I can tell, in most cases, there is no sensible zero value anyway (e.g., what should be the zero value for the type Person?), and you artificially invent one, the only thing library code can do with it is either (0) crash at runtime, or (1) make it impossible in the first place.
A value that inhabits every type is usually a mistake as it makes proving particular things about your program difficult. Haskell has bottom, most other languages have null.
Suppose it depends on what “has bottom” means. I could say that ML reductions might not terminate and then just special case that out of the language semantics. Which I think is more or less what happens—semantics are applied largely to values and values have concrete meanings.
But you definitely have to deal with this in considering the semantics of functions so I suppose you’re right to say there’s not really any escape. Except a termination checker.
<strike>This is not correct. A value is anything that a variable can be substituted with. In a strict language, variables can only be substituted with the result of a finished computation - which bottom by definition isn’t.</strike>
Argh. I misread. I thought you said “every language has a bottom value”.
NULL as giving a certain meaning to addresses is just grand. Treating it as a valid value at any type is at best a clunky type system and at worst an obvious lie. Struggling to find a way to provide default values at every type is a lost cause. Forcing them to exist is dumb. Especially in light of user-defined types (an open world).
It’s hard to say, because it’s unclear what the alternative would be in every case and how it would be used or misused. The problem with null is not that it’s a bad idea but that it’s so easy to misuse. In Java, you end up having to check every reference for null if you want to write a correct program: there are no non-nullable types.
Null pointers exist because we want a default value for every C type and we use 0 (or, more specifically, the all-zero bit pattern) with the convention that 0/NULL is never an allowable address.
Why would we ever null-initialize a pointer? If I had to guess (and this is a guess) then it would be a throwback to the era when you had to initialize loop counters (stack-allocated memory local to the function call) before you could use them in a for loop. That is, there was a time when for (int i = 0, ...) was illegal and you had to declare int i ahead of the loop. If you’re iterating with a pointer with a non-constant value, then it seems like you’d be pushed to declare (without initialization) a pointer and then give it a value later on.
As @pushcx notes, it’s slightly better (at least for debugging) to get the same value (0) for uninitialized pointer errors than to get a possibly-nondeterministic garbage value. This advantage can fade in optimized code because null-pointer reference isn’t a runtime error by definition but undefined behavior.
The concept of nullness is valuable. For correctness, option types work better. The main deficit of options would be slightly worse performance (wrapping things in structs and reading tags, as opposed to checking pointers for null) but the only case where I see null pointers being preferable, these days, is when you’re shaving clock cycles.
Furthermore, it’s hard to imagine a cleaner interface when dealing with memory allocations (which might fail). If you’re writing a program in a resource-constrained environment and don’t have the luxury of crashing when you’re out of memory, then it seems to me that checking the pointer is more useful than checking a piece of static memory like errno (which we know isn’t thread-safe) or having to pass in a pointer as an additional parameter.
I don’t think null itself was a mistake. It’s convenient for representing empty and unset values, and that’s why languages designed today still include it.
If you think about it, Haskell’s Maybe is really just a more convenient and safer way of wrapping up null values and forcing the programmer to deal with the null value.
The real problem is when programmers make incorrect assumptions about the data they’re using, and I think null pointers are just one part of that.
I think having the … option … of a nullable representation is a good thing. Making pointers nullable by default is a bad thing. Having no way to express a non-nullable reference is a worse thing. Programmers are less likely to make incorrect assumptions about data when more of the properties of that data are encoded into the type system.
Not necessarily. Certainly not a “billion dollar mistake”. Also “null” is not just one thing across all programming languages. The issue is fairly language dependent.
It also has more to do with programmers than language. There are bigger programmer issues under which the problems with null can be found. Fixing those is the bigger payoff and is language neutral.
In a technical sense, yes, but semantically they’re completely different. An uninitialized variable is meaningless outside of you the programmer. Jill the accountant could care less whether or not variable x has a default value of 0 or is null as long as her program performs as expected. An null field in a database has meaning to the user and to you the programmer. A null field to Jill the accountant symbolizes the lack of existence. For example the end date on record. What do you default that to? You could default it to 12/31/9999 (or 9999-12-31 outside the US); that’s what ERP systems like SAP do. Is that correct? No. You and I both know the record doesn’t actually expire on that date; it’s bullshit filler and wrong to display that to the user like they entered it.
You’re not arguing for the existence of null here. You are arguing for the ability to express the state that Option in or Maybe allow you to express. Null is a hack to get around the fact that the type system is incapable of expressing your system properly.
Null in most languages is a hack to work around the fact that their type system can’t express the valid states for their values. It’s like the designers said
Look we know that every value is essentially an ADT or Sum type. But we aren’t going to let you express that in your types. Instead we’ll give you null and hopefully you’ll check for it in all the right places without forgetting. In dynamic languages that’s sort of forgivable since the type of a variable can change almost anytime. But in a staticly typed language it’s like purposely avoiding a perfectly obvious solution because makes your type system more complicated. That’s not even necessarily a bad thing but at least acknowledge it’s a pretty ugly Hack and don’t treat it like it’s an elegant solution to the fact that every type in your system is a defacto Maybe type. We already know what the elegant solution to that is. It’s been in ML for going on decades now.
No, it’s a useful practical concession. I think people are going to realize how pointless calling unwrap().unwrap().unwrap() can get. It also doesn’t impact memory safety.
Undefined behavior on null dereference should become a thing of the past, though.
The great thing about unwrap is that is tells the compiler “I fully accept that this might panic and I’m okay with it.” It isn’t accidental or because you forgot. It might be because you were lazy but that’s not something any type system can fix. Any sufficiently lazy coder will find a way around your type system.
Yes.
It baffles me that this is even a debate.
In case anyone is unfamiliar with it, Yaron Minsky’s writing on the topic is the best coverage I’ve read on it for people who can’t imagine life without null: https://blogs.janestreet.com/making-something-out-of-nothing-or-why-none-is-better-than-nan-and-null/
The only thing I would add to that post is that static types are not required to gracefully avoid nulls. Erlang by convention returns values from operations that may fail wrapped in a tuple like
{ok, Value}which forces you to pattern-match in order to unwrap the tuple. This indicates that you need another clause in your pattern match to catch non-okvalues. (Though of course in Erlang it’s permissable to fail to handle the error case and just let your process supervisor tree respawn the process.)There’s nothing stopping communities from adopting safer conventions in languages like C or Java.
The thing that makes Erlang “better” is that the conventions have already been adopted broadly, where as in C, or Java there’s still a pretty long, very steep hill to climb.
Java added an Option type in SE 8, and, of course Guava had one prior to that – a convention worth adopting for sure.
Java added an Option type in SE 8, and, of course Guava had one prior to that – a convention worth adopting for sure.
But you still need null checks (or have appropriate annotations I guess) in every method that accepts
Options, since it is just another object and can be null.Or you can adopt a convention that you don’t pass nulls, and check only at the boundaries or when interfacing with libraries that haven’t adopted that convention. Which puts you in the same place as Erlang.
It’s not the same place as Erlang; the conventions have been part of Erlang’s standard library for decades, and Erlang’s syntax makes it basically effortless to deal with their Option types. Java’s is brand new, and the fact that it’s still nullable means it’s really hard to convince people to use it because it doesn’t offer useful guarantees, so it’s no surprise it’s seen very little uptake.
That’s true. It’s putting lipstick on a pig, or something. At some point programmers need to take some responsibility. If your method returns an Option, maybe you should write enough tests to ensure it won’t return null.
While true erlang’s functional nature and runtime makes it a little less problematic to have your processes die all over the place as well. It’s not just the conventions they built up it’s also the way they built their runtime around the expectation that NPE’s would happen basically all the time.
For sure beam is designed to handle failures gracefully, but are these failures really equivalent to NPEs? Or is there just a tendency to handle errors by exiting the process?
I would say an Erlang process exit and a Java NPE are more or less equivalent. What distinction are you claiming between them?
I don’t know Erlang very well, but my sense was always that the “let it crash” philosophy was more about the uncertainty of running distributed systems–can’t connect/lose the connection? “Let it crash!” Instead of, “Go ahead! Be a sloppy programmer!”
You’re correct; it’s very different. An NPE in Java means “some programmer screwed up” while an Erlang crash usually means “some external situation prevented us from proceeding, so we should try again”.
Both can cover both meanings though. A java program can deliberately allow a RuntimeException to propagate for a transient failure knowing a retry will happen at high-level - I’ve worked on an application that was structured like this (basically a pipeline of queues, if processing a given message failed we’d retry it a few times and then error it out). And an Erlang crash can be a programmer screwup.
I bet you were expecting a yes or no answer, but my answer is null.
:-)
Do you mean that “Not Applicable”, or that the answer is “Not Found” in your brain, your brain is “Not Initialized”, or your brain is in an error state, or you haven’t thought about it yet? :-D
Funny, it’s almost as if representing a number of disparate failure conditions using a single value results in a lot of ambiguity and easily leads to misunderstanding. Who would have thought!
Null makes sense in a type system that lacks sum types and a way to require pointers be initialized.
For example, the closest thing to sum type column in SQL is to have multiple tables with foreign keys to the record you’d want to have the column on, plus a constraint. Rather than every
blog_posthaving a nullablepublished_atdatetime, have a tablepublishingwith an FK pointing atblog_post, anatdatetime, and a constraint specifying exactly 0 or 1 of them exist. That’s a lot of hoops to jump through (extra records, thinking through a join instead of just sayingpublished_at is null, etc.) and implies something that might not be part of your domain (ablog_postcould be published multiple times, at first glance) to represent something that’s adequately expressed by a nullable type.Or in C++, when the language doesn’t require that
int * fooget initialized to a validintbefore it can be passed around and dereferenced. There’s something in there, right? Having a concept for null strikes me as marginally less bad than passing garbage values around without a name, but I dunno.I’d put it on the big pile of programming concepts that seemed like a decent idea at the time but are really problems better solved differently.
(Also for context, this topic came out of chat about this story.)
[Comment removed by author]
No. Null/zero values are a useful, safer alternative to uninitialized values. The mistakes are 1) bad type systems that don’t properly account for null 2) programming with raw memory structures directly most of the time and 3) pervasive library code that don’t do sane things with zero values.
Usually when I revisited code where I have set a pointer to “null”, at the end of the refactoring all “nulls” have magically vanished.
I can’t say I can think of a case where after refactoring I thought, gee, I’m so glad I have null available.
I often see (and have on occasion used) null as an overloaded back channel to communicate “not found”.
Usually with a bit of deeper thought I refactor to smaller functions, one of which returns a boolean “not found”.
Usually with a bit deeper thought I use some more functional approach and even that often disappears.
What would you say is the right way to deal with null? Force nullable pointer users to branch on whether the pointer is null? That seems just like options, only without the convenience of options being opt-in. As far as I can tell, in most cases, there is no sensible zero value anyway (e.g., what should be the zero value for the type
Person?), and you artificially invent one, the only thing library code can do with it is either (0) crash at runtime, or (1) make it impossible in the first place.A value that inhabits every type is usually a mistake as it makes proving particular things about your program difficult. Haskell has bottom, most other languages have null.
Notably, strict functionally-types languages have no such values. ML comes to mind.
[Comment from banned user removed]
Agda/Coq haven’t solved the halting problem and neither have a bottom.
Actually Agda/Coq have solved the halting problem because they are total languages (not turing complete).
They don’t have the Halting Problem. It’s not solved, just non-existent.
It’s possible to do Turing-Complete programming using Agda and Coq using the Partiality Monad:
https://github.com/agda/agda-stdlib/blob/master/src/Category/Monad/Partiality.agda
But then you have the halting problem again, of course.
Suppose it depends on what “has bottom” means. I could say that ML reductions might not terminate and then just special case that out of the language semantics. Which I think is more or less what happens—semantics are applied largely to values and values have concrete meanings.
But you definitely have to deal with this in considering the semantics of functions so I suppose you’re right to say there’s not really any escape. Except a termination checker.
<strike>This is not correct. A value is anything that a variable can be substituted with. In a strict language, variables can only be substituted with the result of a finished computation - which bottom by definition isn’t.</strike>
Argh. I misread. I thought you said “every language has a bottom value”.
Every time I have taken CJ Date at his word, and purged my SQL of every NULL….
..it has become simpler, faster, more maintainable and lost a few subtle bugs along the way.
https://www.youtube.com/watch?v=kU-MXf2TsPE
NULL as giving a certain meaning to addresses is just grand. Treating it as a valid value at any type is at best a clunky type system and at worst an obvious lie. Struggling to find a way to provide default values at every type is a lost cause. Forcing them to exist is dumb. Especially in light of user-defined types (an open world).
It’s hard to say, because it’s unclear what the alternative would be in every case and how it would be used or misused. The problem with null is not that it’s a bad idea but that it’s so easy to misuse. In Java, you end up having to check every reference for null if you want to write a correct program: there are no non-nullable types.
Null pointers exist because we want a default value for every C type and we use 0 (or, more specifically, the all-zero bit pattern) with the convention that 0/
NULLis never an allowable address.Why would we ever null-initialize a pointer? If I had to guess (and this is a guess) then it would be a throwback to the era when you had to initialize loop counters (stack-allocated memory local to the function call) before you could use them in a
forloop. That is, there was a time whenfor (int i = 0, ...)was illegal and you had to declareint iahead of the loop. If you’re iterating with a pointer with a non-constant value, then it seems like you’d be pushed to declare (without initialization) a pointer and then give it a value later on.As @pushcx notes, it’s slightly better (at least for debugging) to get the same value (0) for uninitialized pointer errors than to get a possibly-nondeterministic garbage value. This advantage can fade in optimized code because null-pointer reference isn’t a runtime error by definition but undefined behavior.
The concept of nullness is valuable. For correctness, option types work better. The main deficit of options would be slightly worse performance (wrapping things in structs and reading tags, as opposed to checking pointers for null) but the only case where I see null pointers being preferable, these days, is when you’re shaving clock cycles.
Furthermore, it’s hard to imagine a cleaner interface when dealing with memory allocations (which might fail). If you’re writing a program in a resource-constrained environment and don’t have the luxury of crashing when you’re out of memory, then it seems to me that checking the pointer is more useful than checking a piece of static memory like
errno(which we know isn’t thread-safe) or having to pass in a pointer as an additional parameter.No, it’s a reflection of the fact that programs run on actual computers.
By this logic, shouldn’t we be allocating registers by hand still?
I don’t think null itself was a mistake. It’s convenient for representing empty and unset values, and that’s why languages designed today still include it.
If you think about it, Haskell’s Maybe is really just a more convenient and safer way of wrapping up null values and forcing the programmer to deal with the null value.
The real problem is when programmers make incorrect assumptions about the data they’re using, and I think null pointers are just one part of that.
I think having the … option … of a nullable representation is a good thing. Making pointers nullable by default is a bad thing. Having no way to express a non-nullable reference is a worse thing. Programmers are less likely to make incorrect assumptions about data when more of the properties of that data are encoded into the type system.
Not necessarily. Certainly not a “billion dollar mistake”. Also “null” is not just one thing across all programming languages. The issue is fairly language dependent.
It also has more to do with programmers than language. There are bigger programmer issues under which the problems with null can be found. Fixing those is the bigger payoff and is language neutral.
Sort of. I think the lack of non-nullable types is the real mistake.
In SQL NULL isn’t a type, it’s a state. So it’s not really fair to compare NULL in SQL to null in, say, python.
Hmmm? Isn’t
x= Nonejust the “state” of a location in memory namedx? How is it dissimilar to the location (table, row, column) beingNULL?In a technical sense, yes, but semantically they’re completely different. An uninitialized variable is meaningless outside of you the programmer. Jill the accountant could care less whether or not variable x has a default value of 0 or is null as long as her program performs as expected. An null field in a database has meaning to the user and to you the programmer. A null field to Jill the accountant symbolizes the lack of existence. For example the end date on record. What do you default that to? You could default it to 12/31/9999 (or 9999-12-31 outside the US); that’s what ERP systems like SAP do. Is that correct? No. You and I both know the record doesn’t actually expire on that date; it’s bullshit filler and wrong to display that to the user like they entered it.
You’re not arguing for the existence of null here. You are arguing for the ability to express the state that Option in or Maybe allow you to express. Null is a hack to get around the fact that the type system is incapable of expressing your system properly.
@zaphar, maybe… but I feel like a rose by any other name is still a rose. The SQL programmer would argue that NULLs for him/her are just as annoying.
Sure. And I would argue that the same principle is at play. Your RDBMS is not giving you enough expressive power in it’s type system.
Null in most languages is a hack to work around the fact that their type system can’t express the valid states for their values. It’s like the designers said
Look we know that every value is essentially an ADT or Sum type. But we aren’t going to let you express that in your types. Instead we’ll give you null and hopefully you’ll check for it in all the right places without forgetting. In dynamic languages that’s sort of forgivable since the type of a variable can change almost anytime. But in a staticly typed language it’s like purposely avoiding a perfectly obvious solution because makes your type system more complicated. That’s not even necessarily a bad thing but at least acknowledge it’s a pretty ugly Hack and don’t treat it like it’s an elegant solution to the fact that every type in your system is a defacto Maybe type. We already know what the elegant solution to that is. It’s been in ML for going on decades now.
Duh!
No, it’s a useful practical concession. I think people are going to realize how pointless calling unwrap().unwrap().unwrap() can get. It also doesn’t impact memory safety.
Undefined behavior on null dereference should become a thing of the past, though.
I’ve written over 40 Rust crates and I’ve never written a single
unwrap().unwrap().unwrap().I have written
unwrap().unwrap()20 times though. 14 of those were in tests.The great thing about unwrap is that is tells the compiler “I fully accept that this might panic and I’m okay with it.” It isn’t accidental or because you forgot. It might be because you were lazy but that’s not something any type system can fix. Any sufficiently lazy coder will find a way around your type system.