1. 13
  1.  

  2. 23

    null is not the issue” followed by replacing it with something that is part of the domain model which represents the fact that the value is not defined is pretty much what I would call “null is exactly the issue”.

    I don’t think anybody was arguing that null/Nothing/None represent inputs for which no meaningful operation is possible, these states are often unavoidable.

    1. 12

      The article describes a problem I have seen multiple times in real-world programming companies. I think people cargo-cult the idea that null is bad and decide to eliminate null without understanding why it’s bad, and end up reintroducing most of the problems they previously had with null.

      1. 6

        (I’m the author) Hi, that was exactly my point :-) people just read that null is bad and go on to even worse solutions (like the NOP). I’ve chosen to write examples in java because it’s the target audience. Optional has a lot of issues, but at least it’s a marker and it’s in the stdlib, so it’s a good first step for people new to this kind of things.

        1. 5

          Your article does not actually include samples or discussion of the Null Object Pattern. If you’re thinking of this code sample, this is the sentinel value antipattern:

          // Turbo-bad
          String notThere = ""; // or "N/A", etc
          String there = "my string";
          
          1. 1

            You and I and probably everyone in this thread know that, but there are a substantial number of people who would call that “the null object pattern”, and there’s not (to the best of my knowledge) a well-known definition or blog post that one can link to to clearly explain the distinction.

            1. 1

              I didn’t include a complete NOP example to keep things short. I’ve put a link to the wikipedia page for those interested. Regarding the article goal about having a proper domain model, sentinel value and NOP are in the same problem: not representing missing values at a structural level.

              1. 1

                It depends on the domain though. Sometimes a null object is a legitimate way to represent absence (e.g. there are cases where an empty list correctly represents that a user hasn’t added anything to that part of their profile)

      2. 3

        Add Optional<T> to that list, since people often mistaken that for not being null in disguise.

      3. 7

        Suggesting the use of Java’s Optional is a complete disaster.

        I’m not sure how the designers in Java 8 took a valid idea and could ruin the very substance of it so thoroughly as they did with Optional. This was Java’s “we have had enough of experts” moment.

        1. 2

          Can you explain how Optional is so thoroughly ruined? I don’t write much Java so I haven’t really experienced it. I am an Ocaml user though.

          1. 13

            It breaks both Functor identity and composition laws.

            nonEmptyOption.map(x) // possibly empty Option
            map(x).map(y) != map(x.andThen(y)
            

            The broken null handling makes it rather useless for the exact issues they are now struggling with in Java 10, the famous “what are we going to do with Map#get for value types???”

            Disallowing null as a value inside Optional exposes a really really scary and disturbing misconception some people have about the purpose of this type: It’s as if they said

            using null as some kind of special error value is bad, so let’s invent Optional … and reuse null as a special kind of error value!

            WAT?

            1. 4

              using null as some kind of special error value is bad, so let’s invent Optional … and reuse null as a special kind of error value!

              That’s a wonderful summary!

            2. 3

              Oddly, one of the criticisms is that it can never contain null :). (which, for example for Maps, makes a lot of sense, if the Optional expresses the presence of a Mapping!)

              https://developer.atlassian.com/blog/2015/08/optional-broken/

          2. 6

            I didn’t find much interesting in this article. The idea to use an option type to model the possible absence of a value is well-known from languages that support sum types and probably very familiar to lobste.rs’s audience.

            Also, from the safety perspective Optional in Java arguably makes things even worse: now you have to check whether a value is present in addition to doing a null pointer check (since null is still a valid value for a variable with the type Optional).

            1. 3

              Optional in Java provides a migration path. It becomes possible to enforce “no nulls except at the boundaries with external libraries”, and then eventually perhaps you can switch to libraries that don’t have null anywhere and deprecate it entirely.

            2. 4

              The domain model is absolutely the problem. Spot on.

              Sometime the model is so simple that null is a fine solution. If the implementation language has Optional then just as good if not better.

              Many times the problem is more than “missing or not”. The domain model often must account for, and distinguish among:

              1. Missing
              2. In-progress but it’s not ready yet
              3. Malformed – we’ve got it, but there’s a problem with it
              4. Secured – we’ve got it, but you cannot see it
              5. And so on

              The billion dollar mistake, if there is one, often is in the analysis and design of the system. Mapping the design to a language that does or does not have null or some other mechanism is usually a secondary problem and should rarely if ever keep someone awake at night due to crashes or exceptions or other unexpected behavior.

              1. 2

                I agree with you, and this author, that the domain model is the issue.

                I’ll add that even when null is an appropriate solution for the immediate problem, it is bad for code health in the long term.

              2. 3

                I think null gets a bad rap, but I don’t think the solution is to call it Option.

                In K we often use null to describe “holes” in data, rather than as a generic condition variable (like an error):

                  "<>"?"hello <world>!"
                0N 0N 0N 0N 0N 0N 0 0N 0N 0N 0N 0N 1 0N
                

                (The word 0N refers to a null integer). This representation might be similar “NaN” in C and C++ (and maybe other languages), and grants us certain special effects. Indeed, for many operators, f(x,y) returns null if either x or y are null and f doesn’t specifically deal with this situation.

                Smalltalk takes this familiar approach with nil which consumes all messages, returning nil. An operation can specifically check for nil, but this is exceptional.

                You might also note that SQL gives NULL a similar treatment: Distinguishing between an empty relation, a relation containing NULL, and an error condition is something more languages should do.

                1. 7

                  Option is not just a renaming – it gives you a way to distinguish at type level between values that can be null (you didn’t check yet) and values that can’t (the check has been performed).

                  1. 1

                    Yes it really is, or rather: I think it is useful to think of it as such.

                    The NULL that Option is wrapping as part of its subtype isn’t different from the NULL outside of it: It’s used to mark the same kinds of things, and to refer to the same kinds of things. It’s a poor-mans exceptional condition, and that is the real problem.

                    On the other hand: 0N isn’t exceptional in K; NULL isn’t exceptional in SQL. These are clearly not the same thing as NULL in C or C++ or Java or Ada, so when we talk about real alternatives to NULL I think it’s much more useful to talk about them than about Option.

                    Also: We’ve had value or range-restricted types for a long time, and I do not believe there’s any real utility in them as a concept distinct from any other condition check. I feel the same cognitive load, and using assert() is potentially more flexible.

                    1. 6

                      Neither SQL NULL or 0N in K are statically distinct from non-nullable values because. SQL does not offer non-nullable types. In this languages which offer Option as part of a consistent plan of value-typing – Haskell, Swift, Scala – it gives you a way to opt in or out of nullity.

                      Adding Option to C might not accomplish very much, for example; other than serving as documentation of programmer intent.

                      1. 1

                        Neither SQL NULL or 0N in K are statically distinct from non-nullable values because SQL does not offer non-nullable types.

                        What exactly do you mean by “statically distinct?”

                        K absolutely has a generic null atom (it’s spelled ::). Nobody uses it because it’s stupid (which we agree on), but K offers something better that is more general than “option types”.

                        In this languages which offer Option as part of a consistent plan of value-typing – Haskell, Swift, Scala – it gives you a way to opt in or out of nullity.

                        No it doesn’t.

                        If you have a routine getUserByName that takes a name of a user and can return the user if they exist, the question is what do you do if they don’t exist:

                        • You can return a special sentinel value. You can call it NULL or you can call it Option. I can call it Benny if you want.
                        • You can return a special proxy-user which looks and smells like a user, but they don’t have a name (Smalltalk) or any other meaningful attributes
                        • You can return an empty list. This is useful if you are in an array language (like K or SQL) because you don’t generally return a user, you return a list of users.
                        • You can look-up the condition handler table to find out what the programmer wants returned in this situation (CL)
                        • You can throw an exception, performing a dynamic go-to to some other place

                        The “option type” is obviously the same as NULL. What it offers is a clue to the caller to add more code, which people who want to use NULL are assuming is necessary: That assumption is what is wrong, and it’s what’s preventing you from realizing or appreciating what the relational approach actually wins you.

                        The “empty list” approach is usually the correct one. It produces the smallest and simplest and most-consistent code for the caller, and nobody will actually be surprised by its results, however it isn’t how most C++ programmers program, so even if it’s smaller and better and faster, it can still be culturally unacceptable. So it goes.

                        1. 2

                          I want to suggest that you have a misunderstanding of option types given that you unify them with special sentinel values. They are very, very much more similar to returning an empty list.

                          The trick is that you have a higher order type constructor (Maybe, Option, whatever) which adjoins a sentinel value to a perfectly normal type and operates parametrically on it. This both (a) ensures that the nullity property is reflected statically so that nobody can forget to make a check and (b) ensures that code which works on the underlying type continues to work by assuming success and being mapped through the maybe type.

                          In other words, sentinel values don’t let you truly separate failure cases from success cases. They end up mixed and all code which operates in that area has to complect those two behaviors.

                          As a test, consider the type Maybe (Maybe a). If you’re using sentinel values then you can’t distinguish between each “layer” of failure here since they all get squashed into your sentinel. This is a major failure if the means of errors differ. You can solve it through the existence of multiple sentinel values, but I hope there’s no argument that such a technique scales well.

                          On the other hand, List (List a) does just the right thing when considering failure cases at empty lists: you can always operate over the values which might exist in exactly the same way no matter how many levels of List you slap on top.

                          This is fundamentally the importance of the programmatic notion of Functor.

                          1. 1

                            As a test, consider the type Maybe (Maybe a). If you’re using sentinel values then you can’t distinguish between each “layer” of failure here since they all get squashed into your sentinel. This is a major failure if the means of errors differ. You can solve it through the existence of multiple sentinel values, but I hope there’s no argument that such a technique scales well.

                            This only occurs if you require a sentinel value to be a unique atom. As soon as you remove that restriction (for example using range-restricted subtypes, as I suggested), they’re clearly exactly the same: One can meaningfully perform three operations on Maybe a: Match, Return, and Encapsulate. These are exactly the same operations you can meaningfully perform on NULL or any other exceptional object.

                            I want to suggest that you have a misunderstanding of option types given that you unify them with special sentinel values.

                            Okay, but try to imagine it from my point of view:

                            I’ve been programming for around thirty years, and have professional experience in languages with option types like ML and Haskell, and professional experience in array languages. I think this is something I discovered relatively recently, and I’m trying to share it.

                            And it sounds to me like you’re saying if I disagree with you, I must not understand option types. Maybe you think option types are the best way to handle errors.

                            If you are actually wrong, and there’s a programming concept better for error handling than option types, how will you ever discover them?

                            Maybe you just believe they’re better than null: However the difference between one inch and two inches is not significant when we’re trying to talk about something that is different than a mile.

                            When someone wants to talk about the sameness of apples and oranges, even when they are not the same, we can understand what the other person is talking about when we try to imagine how they are the same. This is why I’m talking about operations that you can perform, instead of trying to write a wikipedia article on option types: I don’t want to talk about option types or “null”, I want to talk about something that is different than option types and “null”.

                            1. 1

                              Hey, thanks for following up. It seems like my feeling you misunderstood came from my own misunderstanding. Apologies there!

                              Further, to be clear, I don’t want to imply that if you disagree with me that you misunderstand anything. I suggested that because as I was reading what you were writing it sounded like a misunderstanding independent of whether it ended in agreement with me.

                              So anyway, with that out of the way, let me try to recapitulate what I understand here:

                              • You’re suggesting that NULL as used in SQL and K is of a different quality than null as used in, say, Java
                              • You’re suggesting that Option/Maybe don’t really help here and may be not significantly better than Java-like null
                              • You’re suggesting that Option/Maybe are similar to range-restricted types

                              Then, the heart of what you’re aiming at is that there’s something special in the way that SQL/K handle 0N and NULL which makes them very nice. The principle example of this is the way that these nulls “absorb” everything so that f(… null …) = null for pretty much any function f, similar to NaN.

                              Am I getting close to what you’re talking about? If so, why do you suggest that NaN-style exceptional values are better?

                              1. 1

                                I appreciate that. I’m happy to keep trying if you are, since I would like a world with better programs too :)

                                You’re suggesting that NULL as used in SQL and K is of a different quality than null as used in, say, Java

                                I don’t know exactly what you mean by “quality.” I don’t know if it’s important.

                                You’re suggesting that Option/Maybe don’t really help here and may be not significantly better than Java-like null

                                I don’t know exactly what you mean by “here”. Option/Maybe doesn’t make programs shorter. It may make it easier to make programs correctly, but I don’t see any case where they can help over better tooling, and I’m not looking because there’s a simpler solution. Tooling is also hard work.

                                You’re suggesting that Option/Maybe are similar to range-restricted types

                                Yes. Rather better to invert this though: In the way that Option/Maybe are similar to range-restricted types…

                                Then, the heart of what you’re aiming at is that there’s something special in the way that SQL/K handle 0N and NULL which makes them very nice. The principle example of this is the way that these nulls “absorb” everything so that f(… null …) = null for pretty much any function f, similar to NaN. Am I getting close to what you’re talking about? If so, why do you suggest that NaN-style exceptional values are better?

                                You’ve skipped a step: I’m suggesting that not having exceptional values is better.

                                0N is not an exceptional value; NULL in SQL isn’t exceptional. (nil in Smalltalk can be exceptional simply because people say isNil but this is up to the programmer…)

                                null is an exceptional value, and as a result has to be checked. All option types give you is the ability to encode that this check has occurred, and cheaply to enforce that it has been checked. When the code is correct, it is still roughly the same length.

                                However: If your program is designed to not have exceptional values, it will be shorter.

                                Maybe if you fully consider the example of trying to load a user from a main database and an impersonation database, you can use a match, or you might be tempted to reach for sequence or catMaybes but an array programmer can simply use join because it is defined for all values.

                                Or maybe if you see a relationship between Maybe and List maybe you can imagine what life might be like if every operation was generalised for a List, then you could see that you didn’t need Maybe anymore.

                                1. 1

                                  First, the mechanical points

                                  • By “quality”, I didn’t mean to give it a definition. It’s merely a marker to denote difference.
                                  • I’m definitely not fundamentally interested in making programs shorter. Only if that achieves another goal.
                                  • I’m not sure Maybe/Option can be replaced by better tooling. I actually think it can’t even come close.

                                  Anyway, to the heart:

                                  I’m suggesting that not having exceptional values is better

                                  I’m reading “exceptional” to (a) imply the existence of a “normal” order of things and then (b) some set of possible divergences from “normal”. I’m reading “value” in the standard theoretical math/CS way.

                                  I’m not familiar enough with K to say, but I can say with some confidence that it’s easy to interpret SQL NULL as exceptional. It’s not exceptional in the sense that it cannot be handled, that it represents failure, but if I say that I have a relation between Persons and their Ages then it is a different concept when it’s a relation between Persons and their (Ages + Null)—an exceptional value has been adjoined. It might take a perfectly banal interpretation (“we don’t know this person’s age”) but in relation to the “normal” order of Age = Natural Number, it’s an exceptional departure.

                                  So, maybe this is running long now, but I really can’t walk this line without that above assumption: “exceptional” needs to be defined against something non-exceptional. Another example is the situation where exceptions (really: non-local returns) are used to simulate early loop termination instead of a genuinely “exceptional” situation.

                                  But with that aside, I do definitely see where you’re going with the last bit: what if everything were a List/Array/WhatHaveYouSequentialThing? Well, we’re now working in a domain which has a built-in exceptional value: emptiness.

                                  Why do I call this exceptional? Well, first it comes from an assumption that the normal order of things is for “substance”. The lack of substance is exceptional. Without that idea, there is nothing exceptional—I agree—but we should take care before throwing away such a nice mental model.

                                  There’s also something really important about the empty array in terms of the domain of arrays: it has the right algebraic properties for an exceptional value. In particular, Arrays seen as monoids have empty as their zero. Arrays seen as monads have empty as a “failure” value (which has a mathematical definition I’m eliding, but shouldn’t be thought of as similar to the “exceptional” term of art I’m using).

                                  The question would then be: how does it feel to work in a world which takes a single notion of failure as basic and includes it everywhere? Well, it’s probably going to be very nice to handle failures.

                                  That’s my current understanding of what you’re talking about. If you work in the computational context of the list monad then you can have a failure effect anywhere. What’s also really nice is that once you interpret your domain as a value domain instead of computational context domain you don’t have to see the exceptional value as being “outside” your domain—you’ve shifted your perception to enlarge what’s “normal”.

                                  But I actually don’t really like that at all. It’s similar to what I said earlier, not even knowing I’d end up here: I don’t really care to optimize the shortness of code. Instead I aim to increase the clarity and legibility of code.

                                  To that end, Maybe is crucial for a non-obvious reason. It’s not what happens when you use it but instead what happens when you do not: absence of exceptionality.


                                  So, that’s what I’m currently getting what you’re aiming at here. I think I understand your idea of non-exceptional values both from my experience with SQL and, interestingly, Matlab.

                                  I also think it’s really convenient.

                                  But I think it’s less legible. It leaves me less able to say clearly the things I’d like to say unless they fit into a world with a built-in failure notion. I don’t always want to do that.


                                  As a final idea, I am very willing to consider Maybe and List as being closely related. I’d say from a certain perspective they’re essentially doing the same thing. Normal execution results in exactly one value. Maybe suggests computation which might return 0-or-1 values. List extends that to 0-to-many.

                                  I think any semantic advantage related to failure arising from taking Lists as your computational context translates immediately, directly to Maybe values. There are definitely syntactic improvements available when you’re working in a more specific computational context. There are also lots of mental overhead improvements. If you don’t want a general language, don’t ask for one, I suppose.

                                  1. 1

                                    I’m definitely not fundamentally interested in making programs shorter. … Instead I aim to increase the clarity and legibility of code.

                                    I do not make poetry or tend bonsai. If the program produces the wrong result, I don’t care how “clear” or “legible” other people believe it is.

                                    I want my programs to be fast and correct and I want to get them done as quickly as possible.

                                    To that end, shorter programs are faster (smaller binary) and do have a higher probability of being correct (fewer characters).

                                    This is not subjective, and I’m willing to entertain anybody who tells me they have written something shorter or faster than me, and I am interested to hear from anyone who says they have written something shorter and faster than me.

                                    But that you can find the colour blue “clearer” is not something I’m comfortable having a conversation about: We simply can simply disagree on whether the program is clear or not, and maybe this is a popularity thing where more people think it is less clear than I do. Yet if I have a lower defect count, this is not important if the purpose is to increase the “clarity and legibility” of code at the expense of correctness and time. No thanks.

                                    The question would then be: how does it feel to work in a world which takes a single notion of failure as basic and includes it everywhere? Well, it’s probably going to be very nice to handle failures.

                                    I don’t think this is right. The better thing is to simply not introduce a “notion of failure”. Start by trying to rethink the problem so that you do not carry Maybe or null more than one function from its source, and then re think again so that you simply don’t produce Maybe or null in the first place.

                                    I can say with some confidence that it’s easy to interpret SQL NULL as exceptional. It’s not exceptional in the sense that it cannot be handled,

                                    I’m not sure I understand this position. SQL NULL can be used as an exceptional value. It is not (in practice) used as an exceptional value.

                                    An exceptional value is simply a value that represents an exceptional condition. If you introduce, by return or by throw that condition into your program, you must handle that condition or your program is not correct. Again, it’s not subjective or negotiable, the practical issues are that you have an error if the programmer makes a mistake and:

                                    • Java null: if the condition occurs
                                    • Maybe/Option: if the condition is not checked
                                    • Smalltalk nil: in no other case
                                    • Empty: in no other case

                                    (and I still don’t see why Java compilers can’t trace Java null by converting it into a maybe inside the AST and perhaps class file…). Yet while it is clear that the programmer is most likely to have trouble in the Java case, it may be less clear that the programmer is simply less likely to make a mistake in the Empty case than in other cases, so it is better.

                                    1. 1

                                      So I think we’re at a fundamental difference now. I’m very interested in “clarity” and I measure it in terms of the logical or denotational properties of fragments of my code. It’s not a specious or subjective or poetic thing at all. It is a matter of comparative linguistics, but I think we can’t easily find something better.

                                      On the other hand, I really strongly disagree that code that is shorter is more likely to be correct. I also feel that it’s more likely to be difficult to change over time incurring large costs to the learning cycle.

                                      But let’s drop all of that because while it’s interesting, it’s not the matter at hand.


                                      I like your definition of exceptional value—it’s clear what you mean—although I am not sure I agree with it. Let’s run with it for a second to be sure I understand what you’re saying, though, and then we’ll get back to why I disagree.

                                      What I’m reading out of your definition and discussion of exceptional values is that (a) they arise from some process and (b) they demand handling. I feel like you’re suggesting that if an exceptional situation arises (you mention from programmer error, but also it might be a natural, expected situation like a file not existing) then it is an error if they are not properly attended to.

                                      Then, from here, you note that Java nulls are an error immediately because there’s nothing to signal the need to attend, Maybes are errors if they are not pattern matched on, but Smalltalk nil and Empty never need handling and therefore are more concise, lower energy, and better.


                                      So, I think the core of my disagreement is that exceptional values need to be handled. In particular, I often separate the “computation” where an exceptional situation might arise from the “interpretation” of that computation elsewhere in the program. In this case, you might do more analysis of the exception than just identifying whether or not it has happened. For this reason, signaling the existing of errors in types is key since it allows you to discuss the transition from an error-creating computation to an error-free one.

                                      Fundamentally, that is the single reason why Maybe is valuable and, in my opinion still, much better than the others.

                                      I think often exceptional values might arise in one context, but in their interpretation elsewhere they become no longer exceptional. A good example might be a search algorithm which “blows up” when it fails to find its target. From the search’s POV there is nothing further to do and exceptional results are necessary. From the interpreter’s (or “user’s”) POV, we just learned that the search failed. Maybe this is what we wanted!

                                      Obviously, nil and Empty handle this too, but they do so by inhabiting all types (I think). That means that there’s no longer any static information to guide the semantics of the program.

                                      So back to the first point in this response: we may just not have the space to agree here since we have different values on what makes a program good. That’s okay, and I appreciate the discussion. I also feel like I do understand your point here even if I don’t personally want it.


                                      As a final note, the reason Java can’t do static analysis to detect nulls is that a lot of code has bugs in it were it to be passed what may-as-well be a null but actually isn’t ever. This would mandate many more checks than people would ever want to do (even if we upgraded the AST to also fmap and bind most of these operations so that we’re living in the Maybe context).

                                      As an example you can look at how people in the typed javascript world are doing here. You can compare the Typescript world to the Flow world. The former decided initially not to track nulls and made for very easy interop. The latter tracked nulls and made for a much better programming experience, but also gets crushed on interop sometimes. In both cases, too, I think they use a notion of null that’s more implicit than Maybe—more like your Smalltalk nil.

                                      1. 1

                                        I feel like you’re suggesting that if an exceptional situation arises (you mention from programmer error, but also it might be a natural, expected situation like a file not existing) then it is an error if they are not properly attended to.

                                        If a program needs a file to exist, and it doesn’t (yet) exist, printing out a message and waiting for the file to exist might be a perfectly sensible way to deal with it. Note in this situation no “error value” is needed because the function simply does not return until it has been successful.

                                        The problem “of null” shows up when a programmer says “not my problem” and returns an exception. They are expecting the caller to notice this exception, print the error message themselves, and then retry. Or not. The problem doesn’t go away because the programmer, saying “not my problem” used Maybe instead.

                                        I think often exceptional values might arise in one context, but in their interpretation elsewhere they become no longer exceptional. A good example might be a search algorithm which “blows up” when it fails to find its target.

                                        If the object is expected to exist, then yes it would still be an error.

                                        Whether you write Search(Tree,Object).Doit() or Tree[Object].Doit() makes no difference.

                                        John demonstrated how you can write a search algorithm without if statements or conditions or exceptional values, so it’s actually a very good example of something that you think requires an exceptional value, but really doesn’t when you’re thinking about the problem correctly.

                                        I think programs are better (shorter, faster, more correct) when they don’t use exceptional values. Is this becoming more clear?

                                        So, I think the core of my disagreement is that exceptional values need to be handled. In particular, I often separate the “computation” where an exceptional situation might arise from the “interpretation” of that computation elsewhere in the program.

                                        I don’t understand this.

                                        A human being decides whether your program is correct or not, by whether it does what they expect it to do given the inputs they have supplied it. Collectively, users vote on your programs' correctness, over all of its runs over its lifetime. If it crashes because of an unhandled null pointer, or because of a handled one makes no difference.

                                        When I’m referring to the program being correct, I’m talking about it being judged correct and not by whether it compiles or calls exit(0).

                                        If it is “as specified” then I’ll accept it as being correct for the purposes of moving the discussion to how difficult it is to specify program behaviour, even though I don’t see any difference between that and programming as well.

                                        On the other hand, I really strongly disagree that code that is shorter is more likely to be correct. … But let’s drop all of that because while it’s interesting, it’s not the matter at hand.

                                        This seems completely obvious to me now: I cannot yet prove that program complexity is proportional to problem complexity, but I can find no counterexamples without serious flaws, and I have had a number of experiences that support this strongly: Database languages made 1000x faster just by being shorter, and “hello world” that anyone can write. Every if statement doubles the number of paths your program can take; every pattern match multiplies combinations. Loops? How long is a loop?

                                        AW is reported to have quipped “the only program that has a chance of being correct is a short one” <1>, and I think that is one of the most illustrative ways to think about it: If the program is small enough, and you can fit it fully in your head, then you can fully understand it and appreciate all of it’s possible inputs and outputs simultaneously, then you can convince yourself that it has no bugs.

                                        Anyway, I don’t think it’s absolutely necessary for this discussion to agree that absolutely shorter programs are absolutely more correct if you accept that better programs are faster and more correct. Agreed? We can then agree that fewer conditions/if-statements/branches produce a decrease in program complexity and in that way make it more likely for the program to be correct?

                                        I’m very interested in “clarity” and I measure it in terms of the logical or denotational properties of fragments of my code.

                                        I don’t understand this at all. What are “logical properties of fragments of code”? I think code is letters and shit; things that I type.

                                        As a final note, the reason Java can’t do static analysis to detect nulls is that a lot of code has bugs in it were it to be passed what may-as-well be a null but actually isn’t ever. This would mandate many more checks than people would ever want to do (even if we upgraded the AST to also fmap and bind most of these operations so that we’re living in the Maybe context).

                                        Yes. Put simply, the reason is practical. It can’t do it because it would surprise Java programmers.

                                        However in some alternate universe where Java was still nascent, we could consider the following:

                                        if(foo().bar) { ... }
                                        

                                        as correct if foo() never returns bar, or if the program is supposed to crash when foo() returns bar. There is nothing about maybe that changes this fact, except to declare that the program is not supposed to crash – so if the program is not supposed to crash, then static analysis is enough it determine whether foo() can potentially return null or not: no complicated proof system is required if the program has already been transformed to SSA, just one of the variables will have the type Either(null,T) while the other will have T

                                        When we’re talking about the sameness of null and any other exceptional value, then it makes sense to consider this alternate universe, so that we can properly discuss what would make programming easier.

                                        1. 1

                                          This conversation continues to be interesting and I’ll take a look at that video. I think, however, that the format as a Lobste.rs thread is poor. Shall we continue in email? I can’t find yours immediately, but mine is in my profile.

                            2. 0

                              Yes, this is what I was trying to say.

                        2. 4

                          These work because you’re really thinking about a larger structure—K arrays, I guess, or SQL tables—which allow for missingness. This is the same as having a finite map, e.g., which lacks values at some keys. This is not a problem at all since it’s not first order—NULL can only appear at well known places.

                          The challenge is when NULL can appear anywhere. At this point you’re playing in a minefield.

                          1. 0

                            Ada having range-restricted types is not the same thing as Java having them (which it doesn’t). How many Ada programmers do you know?

                            C# plans to add non-nullable types, which I am very excited about. I would like to see a distinction between nullable and non-nullable types. You can null-check the former to get the latter.

                            Non-nullability in a mainstream language will be great. Rust already has Result<T, E>, which does a great job of encapsulating errors in a value other than null.

                      2. 2

                        I love the way the ruby community solved this problem (with try, present?, blank?,…)

                        user = User.find_by(email: params[:email])
                        user.try(:send_reset_password_email!)
                        flash[:notice] = "If you have an account, an email has been sent to you"
                        
                        1. 2

                          If you’re using Typescript (should you be using it? If you’re writing any non-trivial amount of JS, probably. Super easy to get started), Typescript 2 has a “strict null check” flag that forces you to check for null/undefined values when possibly present. Basically giving you a language-level Optional/Maybe.

                          Typescript 2 is beta, but I’ve had no issues with it so far for a large-ish project.

                          More info

                          1. 2

                            Using a type system that cannot find nulls statically is like wearing a bulletproof vest with a bug old gap over the upper left of the torso.

                            1. 1

                              Nowadays, we know that null is to be avoided.

                              I didn’t receive the memo, and I haven’t been paying attention much. Anywhere I can read about this? (Nogrammer)

                              1. 10

                                TL;DR: Back in the days, when everything was expensive, language designers figured out a cheap way to refer to/represent “no data”: The null pointer. Referring to something means “pointing” to the location where the data lives in memory. A null pointer (which btw might not even point to all-zeros in memory but let’s ignore that) is basically a pointer that refers to nothing.

                                The problem today is two-fold:

                                • In many older languages, this value null is part of any type a programmer defines. It’s there whether you want it or not, whether it makes sense or not. Defined a type “Car”? Values of Car can be null! And because null was kind of “snuck” into the value domain of a type, type systems often can’t tell them apart from other values at compile-time, but just crash at runtime when you try to do something with a value that is null. In many cases you don’t need or want null as a valid value of your type, but it’s there anyway and in a world where 90% of the code out there is using libraries written by someone else you constantly have to deal with it.

                                • Because null was readily available, people actually started using it to represent various kind of errors, and because many languages made it easy to deal with null, it became a widespread phenomenon and made it unlikely that devs would use more “elaborate” ways of dealing with errors. (There are exceptions, but let’s ignore them for now.) This is a huge issue because

                                  • better ways of handling errors often did not get null’s special treatment, so people didn’t adopt them (too verbose/too cumbersome).

                                  • when things go wrong, you often only get a “NullPointerException” with a trace of called methods, telling you that something tried to work with a null. Especially in concurrent environments with mutable data this is a huge issue because the stack trace only tells you what it did when the null occurred, not necessarily where the null came from.

                                  • because the null was unique, it is basically the same value for every type you have, it can’t carry any domain-specific error information.

                                So what are these better approaches to dealing with errors? It’s mainly about

                                • not considering them to be “exceptional”. There are of course errors which cannot be dealt with gracefully, but most errors don’t belong in this category. Things can fail for many different reasons, and that’s the normal way of life.
                                • offering type-safety by separating the “error” value from the “real” value.
                                • offering appropriate information about what went wrong.
                                • offering operations that deal with handling multiple errors in a graceful way.

                                Many newer languages implement this, not by building things into the language, but by encouraging developers to use types defined in the language’s “common” library (or define their own types). There is not one “replacement for null”, but many different types that handle different kinds of values/errors/issues and provide different amounts of information.

                                In one language, Scala, you have the following types in the standard library:

                                • Option – which is useful when there is only one possible error case which doesn’t need further tracking. If you lookup a value somewhere, you might receive a value Option which either expresses “we found the value, here it is” or “we didn’t find the value, we have nothing to show”.
                                • Either – which is useful if you have multiple error conditions. A value of Either either carries a “success” value or an “error” value which you can inspect further. This is useful if you have something like “either I logged in successfully – or the username didn’t exist, or maybe the password didn’t match, please show me what went wrong”.
                                • Try – Is a bit like Either, but is intended to encapsulate and handle operations that “throw exceptions”, e.g. things that think that their issue is exceptional.
                                • Future – Represents that a value might not be available yet. Example: You make a call to an external database. You receive a value of Future immediately, which gives you a “handle” for the result when it becomes available (or the error that appeared) and let’s you add further operations you want to execute on this value/error without having to wait until the database comes back with the actual result.

                                The important thing is that these types are in no way special. These are just a few where many people decided that they are often useful, and standardized on them. Many devs regularly define there own types to handle more specific conditions, and that’s perfectly fine. Exampe: Validation, which is a bit like Either but instead of one error value it “accumulates” all errors that have appeared along the way.

                                I hope this helps! Let me know if I used any jargon you’re not familiar with and I will rewrite it more clearly.

                                1. 2

                                  To me this is a great explanation. Thanks for shining your light on this; I have dabbled with Haskell, but kept wondering what the deal was with null. iirc Haskell has no pre-defined null value, though I guess undefined and bottom come close? I suppose they’re in the same ballpark at least. Types like Either seem familiar from Haskell, as does Maybe (i.e. Nothing | Just a).