1. 9

    A function may succeed. Or it may, for example, fail because of disconnected backend. Or it may time out. And that’s it. There are only two failure modes and they are documented as a part of the API. Once you have that, the error handling becomes obvious.

    I must be missing something because it really feels like there are plenty of other ways for a function to fail. Is this limited to a specific context? If it’s only for infrastructure, it still seems woefully pidgeonholed.

    As already mentioned, classic exceptions are the worst.

    I’m not clear on why they are “the worst”.

    The discussion does hit on something that makes sense to me: think about and document the error conditions. Frankly, if you have that, the methodology of reporting the error becomes less of a hassle. But still, error handling is plauged by the fact that it is often something non-local that is affecting the computation and there is rarely any useful information or language constructs that make dealing with it anything short of a massive chore. (Correcting it usually means interacting or “conversing” with some other entity to gain the knowledge to proceed.)

    1. 4

      I must be missing something because it really feels like there are plenty of other ways for a function to fail. Is this limited to a specific context? If it’s only for infrastructure, it still seems woefully pidgeonholed.

      POSIX is quite a good example of how it could work. Every function can return few possible error codes and that’s it. The idea is that implemeter of the function deals with the complexity and factors all possible error conditions into a small neat set of error codes that makes sense from the user’s point of view.

      The rule here should be: If you don’t know what to do with an error condition, don’t just pass it to the caller. The caller understands the problem domain even less than you do.

      But still, error handling is plauged by the fact that it is often something non-local that is affecting the computation and there is rarely any useful information or language constructs that make dealing with it anything short of a massive chore.

      The point is to use encapsulation for errors as well as for normal functionality. If something non-local causes an error somewhere down the stack, the layer that deals with the thing (and every layer above it) should convert it into an error that makes sense in the local context.

      1. 1

        If something non-local causes an error somewhere down the stack, the layer that deals with the thing (and every layer above it) should convert it into an error that makes sense in the local context.

        When said this way, I understand the point better. I did not get that from the original post. I think that’s a reasonable way to deal with things, although I don’t think it precludes exceptions as the mechanism for doing it.

        1. 1

          True, but exceptions make it super easy to screw it up. Just forget a catch block in one function and the raw low-level exception escapes up the stack. In C/Golang style of error handling you have to at least pass it up manually which will, hopefully, make you consider whether it’s a good idea in the first place.

      2. 3

        (Correcting it usually means interacting or “conversing” with some other entity to gain the knowledge to proceed.)

        That’s why, even though it is relatively heavy-weight for an API, it seems that passing a callback to be called on error is one of the most versatile things you can do. The callback can correct the error and allow the call to proceed or just throw an exception. At deeper level, doing this allows you to interact with context at the point of detection not the point where you express your intention: the initial call that led to the error.

        I think this is the closest we can come to approximating Lisp’s condition system in languages without those constructs.

        1. 3

          Signals and restarts are wonderful things. It’s such a shame no other language or programming system (to my knowledge) has made a serious effort to emulate it, let alone build on it. Callbacks are the best we can do – or what we’re willing to abide – it seems.

          1. 3

            Have you heard of the Zen of Erlang? https://ferd.ca/the-zen-of-erlang.html

      1. 5

        In my experience, there are four types of errors [1]. To summarize, using connect() as the example:

        • it’s a bug—EBADF should not be happening. That it is, is a bug. Once fixed, it should not happen. So EFAULT, ENOTSOCK, and EISCONN all fall under this category.

        • It’s fixable outside the scope of the program—EACCESS, ENETUNREACH and EAGAIN are examples here. Report, and exit, not much else to do.

        • Resource exhaustion, things are probably going bad quickly—EAGAIN might also be this category. ENOMEM definitely is, but that’s not a possible error for connect().

        • Expected and should be handled by the program—ETIMEDOUT, ECONNREFUSED, EINTR (if using signal()), maybe ENETUNREACH could be considered here as well. Things that can normally happen, and there is some overlap with the second category here.

        It’s now a bit simpler—just check for expected conditions and handle; everything else should be logged (and possibly terminate the program, depending upon the condition).

        [1] On that page I list three ways to handle errors. Since then, my thinking has changed somewhat on that but I’ve yet to write it up.

        1. 2

          I like the categorization. It makes easier to think about the errors.

          One thing that comes to mind is: Can we deal with some of those categories automatically? For example, I’ve never seen ENOMEM handled in a reasonable way. While in theory is looks like it can be handled, thigs like memory overcommitment and OOM killer make it futile. Maybe we’ve given up any chance of handling OOM errors back in 1960’s when we’ve replaced static invocation records by call stack. Anyway, maybe returning ENOMEM makes no sense at all. Instead OOM killer should just kill the process. But I never done embedded programming, so who am I to tell?

        1. 0

          This is why I simply love programming with golang. Everything is setup for you to handle errors somewhat properly, without much hassle.

          1. 3

            But when you look at actual real-world code in Golang, what you often see is:

            err := fn()
            if err != nil {
                return err
            }
            

            Even a sane system can be used in silly way.

            1. 2

              At least the snippet checks for an err value!

              1. 2

                That’s ok, if you don’t want to handle it there.

                Worst is:

                output, _ := fun()

              2. 3

                That’s a joke, right?

                Error handling is one of the few things I think go has comprehensively messed up. In no particular order, the builtin errors lack:

                • Internationalization of error messages
                • Derived errors (high-level-failure caused by low-level-failure)
                • Syntactic support for error propagation
                • Distinction between programmer errors and runtime errors (eg Printf returns an error for a bad format string or stdout being closed)
                1. 1

                  Different opinions, man. It’s not perfect but I think it’s pretty good, compared to all the languages I worked with.

                  Btw, internationalisation of error messages is dead simple to implement, even if it is not straight out of the box.

              1. 8

                One pattern that I’ve noticed is that many web developers don’t get rigorous error handling, perhaps because the context in which we’re developing does so much work for us. If you:

                1. Don’t share mutable state between requests
                2. Don’t eat exceptions
                3. Have a transactional data store

                there’s a limited amount of damage that you can do by being lazy with error handling. The worst case is that you fail to make progress in an instance where you could’ve continued past some exception, but you’ll never end up in an inconsistent state.

                1. 4

                  Yes. Also, if anything goes wrong, user will just hit “reload page” button. No big harm done. I think GUI applications are ones that rarely need rigorous error handling.

                1. 2

                  Bug report: looks like broken markdown around the libdill and POSIX connect() links.

                  1. 1

                    Fixed. thanks!

                    1. 0

                      +100

                    1. 5

                      Sounds interesting, might solve a problem I have at work, too bad it’s coupled to github =(

                      1. 3

                        What would you prefer it to use as the underlying storage? (I am trying to understand what people actually want.)

                          1. 4

                            I was thinking of storing everything, including the comments in a git instance, which would work independently of what git frontend you are using, but then I would have to speak git protocol from the browser which sucks. I may have a look at git.js

                            1. 3

                              Looking at git.js documentation :(

                              “I’ve been asking Github to enable CORS headers to their HTTPS git servers, but they’ve refused to do it. This means that a browser can never clone from github because the browser will disallow XHR requests to the domain.”

                              1. 1

                                Anything self-hosted would be viable, but everything on git would be even better, although probably more complicated. We use gerrit at work (which sucks at several levels), and mostly anything third-party is very much disallowed. Maybe you could create an abstraction that would speak Github API to github and git protocol to other servers where this would work?

                                The other possibility could be a sort of optional backend/proxy, so, if the git server doesn’t have CORS, you could spin that optional server.

                                1. 2

                                  After thinking about it some more, there’s a lot that GitHub offers that I would have to reimplement myself. Authentication, for one thing. If it was used in a stand-alone mode in enterprise, some kind of authentication would be still needed. People would probably want SSO. Then there are notifications. GitHub sends you an email when you are mentioned in a bug. I would have to somehow interact with company’s mail server. And so on. This is my hobby project and I don’t really have time to go into that amount of complexity.

                                  1. 1

                                    Sure, makes sense. It’s still a cool project, nonetheless, so, congrats =)

                              2. 2

                                sounds like a job for the backend

                            2. 1

                              The only issue that I have with it is sharing my organization details. Although you could do it manually, I’m always a bit annoyed about this.

                          1. 4

                            I would like to try this out, but I can’t auth the shopping list demo to Github because it wants access to both private and public repos, and those of my employer. I understand this might be a limit on Github’s side (maybe the permissions options aren’t granular enough) but unfortunately that makes it a hard pass for me.

                            1. 2

                              There’s a scope to authorize the app only for public repos. But that in turn would make it impossible to use on private repos. It’s starts to look like GitHub API, as it doesn’t support granting per-repo access, is not designed to support this kind of scenario (application running in brower, impersonating the user).

                            1. 1

                              I can’t seem to get any of the examples to load workout errors.

                              1. 1

                                Replied on the issue.

                              1. 3

                                I’ve wanted something like this for a while - the idea of mixing documentation/checklists and a record of going through them really appeals to me.

                                Not sure about GitHub-as-storage, but I can see how other approaches might be more work. I was expecting it to be more like a wiki (self-hosted, database backend) on first glance, but I’m not sure if that would actually be an improvement rather than just what I’m used to.

                                1. 2

                                  Yes, checklisted documentation is helpful, but also notice the context-collection capabilities via comments on steps. The motivation use case: A person is working on a complex process for several months, then leaves the company. Another person takes over and has to get up speed quickly. By looking at the graph they can immediately understand what have been done, what haven’t been done and what’s blocking what. By looking at individual unfinished step they can read the comments and understand what was already done.

                                1. 3

                                  This looks great. Is this library being written to be part of some large application?

                                  1. 8

                                    I’ve written ZeroMQ and nanomsg once. This is part of my years-long project to make writing such applications tractable. And by that I mean being able to write them without falling into either callback hell or state machine hell.

                                    1. 2

                                      On that topic, what is the status of Nanomsg? Is libdill your main focus, or do you grow these projects in parallel? I’ve watched this projects without using them in practice, but I really like the approach of trying to find the right abstractions and patterns for expressive and efficient network programming.

                                      1. 1

                                        Banal question; libdill? why not just use go?

                                    1. 3

                                      I love this concept.

                                      One challenge is that one can always read all the pages. It would be great flavor if this shipped with like… 20 times as many pages. Things like random letters, accounting statements. So you can also experience the whole “sifting through a lot of stuff” thing, and perhaps landing on an interesting bit somewhere. Maybe even having this actually ship as several distinct sets of books, for example.

                                      EDIT: You might want to check out Her Story for some interesting ideas in there as well. A bit harder to execute upon on paper, but is a very interesting mechanism for non-linear storytelling.

                                      1. 2

                                        I like the idea of adding cruft to confuse the reader. Another option would be to hide different chapters at different physical locations and references would be instruction of how to get to the next chapter. But the it ceases to be a book, of course.

                                        EDIT: I’ve added a comment to README along the lines of your comment. I hope you don’t mind. One modification though. Adding 20x more content isn’t feasible for a printed book. So, instead, the pulp should look as a legitimate content to waste reader’s time by solving nonsesical puzzles etc.

                                        1. 1

                                          It would be great flavor if this shipped with like… 20 times as many pages.

                                          It’s an example of security through obscurity!

                                        1. 4

                                          If we accept the premise that there is a lot of value in the uniform morphology, Esperanto could be an option suited for ASCII (in the orthography with «x» instead of diacritics, of course). Then there is also Toki Pona. Many people prefer just to combine enough separate English words to get the point across.

                                          But I think there is another linguistical problem to consider: naming things in programming is hard because programming is an activity where minor semantic distinctions often matter. Maybe a uniform morphology would help by reducing verbosity and allowing to put more meaningful roots in the name of a given length; but anything general enough to be universally useful would have to be vague enough to be subtly misleaing in the specific cases anyway.

                                          The problem is not just to remember the words — «reading with a dictionary» is a skill older than programming. The problem is that too much details are needed for defining even a single word.

                                          1. 2

                                            Maybe. But you can also look at it from the other side: If morphology is standardized, people, being pattern-loving animals, would try to use the constructs consistently, i.e. try would try to make relationship between “parse” and “parser” be similar to relationship between “scan” and “scanner”. Eventually, the constructs could come to represent something like “design patterns”, something that you can assume to work in some specific way.

                                            1. 1

                                              Well, the design pattern called Factory definitely has its own «-Factory» suffix. And predicates often get an affix of one or two characters («is», «-p», «?»). And Hungarian notation was used.

                                              My fear is that humans are actually too good at pattern matching, so if all you have is «-er», you will get «parser» regardless of whether it tries to parse a prefix or requires a whole-string match.

                                              Do you hope that using a spoken language with a lot of morphological modifiers as a base will affect the culture to create enough new modifiers for smaller patterns? I mean enough to avoid combining any dangerous-to-combine notions. I find this plausible, but not guaranteed; I guess naming things in Esperanto could be a way to try.

                                              1. 2

                                                I don’t know really, but it might be worth a try. At least when a programmer talks to another programmer in person, they use natural language to get through the idea. This is often (at least in my experience) superior to just reading the code. So, maybe, if we were able to take a bit of this person-to-person communication in convey it via the code, it would help to some extent.

                                                1. 1

                                                  Well, in person-to-person communication there is not only different naming (I would be surprised if some structured morphology not used in the variable naming would arise), there are different protocols for manging the level of details. You can get an overview that is not just «N levels deep in the call tree». Sometimes abstractions are also intentionally lossy, which you are usually not allowed to do in code.

                                                  Some things depend on feedback, there is some research into allowing zoom in/zoom out for abstraction levels, but improving state of art in the area of zooming out the programming abstractions would definitely be valuable.

                                                  1. 1

                                                    Yep. That’s why I said “a bit” :)

                                                    1. 1

                                                      My point was also that we currently have more tools for lexical part than for grammatical part. Is morphology still where the best return on effort is? (I honestly do not know)

                                                      1. 1

                                                        I don’t think this would help with the tooling. However, it would decrease the amount of lexical baggage which in turn could help with, say, keeping the learning curve flat, or, maybe, returning to old code years later, remembering just the core concepts and being able to get up to speed immediately.

                                                        1. 1

                                                          I meant tools in a wider sense including conceptual tools.

                                                          Intuitively, a flat learning curve is not something you can achieve in an experiment (the morphology has to be learnt first). So this part is hard to know (getting data from Esperanto taking off only for code identifiers and comments sounds a bit optimistic).

                                                          It would be of course interesting if there were some subset that you could try with moderate effort and then tell a success story.

                                          1. 9

                                            The next step, being code negative in other peoples software repositories via sharing knowledge alone :)

                                            1. 18

                                              The bullet point on my resume that gets the most comments from interviewers says:

                                              Reduced codebase by 110 KLOC (43%), largely by rewriting the Java subsystem in Python. Increased reliablity from 93% to over 99.5%.

                                              1. 3

                                                That’s the very crux of the problem. How would you shared knowledge without writing code? Well, there’s still an option to write academic papers, but given the rift between compsci academia and practicioners of programming I would expect it not to be very efficient.

                                                1. 2

                                                  Well you can talk to people.

                                                  1. 5

                                                    Think about it in memetic terms.

                                                    The idea is a meme. The code is its reproductive organ. The code is ‘useful’ so that it can lure its prey (a living human brain). Once the code is used the idea is repeatedly injected into the brain.

                                                    Compare that to talking to people where the idea is basically let floating in the space to be voluntarily accepted or not.

                                                    The former approach is much more efficient.

                                                    1. 1

                                                      Ideas spread fine on their own. For example I’m about to convince you of this without a single line of code. There’s no need to push things into formal language when they make sense in nonformal language. I don’t need to tell you the steps of how to build a boat for you to realize that some method of traveling over water is good. In fact I’d argue that if I told everyone the exact steps to build a boat most would miss the point about what the boat is for. They’d get caught up in the details and fail to capture the bigger picture.

                                                  2. 1

                                                    English descriptions with formal specifications and/or pseudocode accompanying them in a formalism that’s simple. That was the standard for high-assurance security. It worked consistently so long as the formalism could handle what they were describing. The caveat that the teams needed at least one specialist to train them on and help them with the formalism. If we’re talking CompSci, that could just become another 101 course where people are exposed to a few easy ones.

                                                1. 7

                                                  There was an interesting argument by David Graeber that “doing useful work” is considered a privilege nowadays: “You are a nurse? And you complain about not being paid enough? Shut up, you are at least doing useful work.”

                                                  In terms of software development that would be: “If you want to do useful work, do shut up and be glad you can do it for free as an open source project.”

                                                  1. 22

                                                    It’s interesting to ponder why Google would invite someone with a completely opposite worldview. Not just different, but a perspective that openly calls for the end of all the Googles out there. I have to watch it.

                                                    Edit: Gold nugget in the last final seconds:

                                                    Interviewer: Do you have anything you’d like to ask us [Googlers, marketers, software engineers]?

                                                    Chomsky: Why not do some of the serious things?

                                                    1. 7

                                                      Remember that Google might not equal the Googler or team of them that invited this person. This is a huge company with a lot of different kinds of people. I imagine they bring in many different kinds of people to suite different tastes. It’s also not going to be threatened by someone disagreeing with it given the audience can just shout the person out the door and not invite them again. One or more inviting him probably liked some stuff he said in a movie or presentation. Then, they thought some people might enjoy hearing him speak. The end.

                                                      That’s my default assumption anyway.

                                                      1. 3

                                                        I agree. In fact it would’ve been more notorious not accepting the proposal of his talk.

                                                        1. 2

                                                          Working at Google (but having no idea of the background of this talk) I would very much expect it to have happened like that.

                                                        2. 7

                                                          It’s interesting to ponder why Google would invite someone with a completely opposite worldview. Not just different, but a perspective that openly calls for the end of all the Googles out there. I have to watch it.

                                                          That’s a good way to signal you’re secure in your worldview: freely invite people to challenge it.

                                                          1. 13

                                                            While I agree with you in the general case, I think Google is doing this to placate it’s employees. What better way to dispel animosity than to accept the other side as one of your own?

                                                            1. 9

                                                              (kind of tangential, but…) I’ve always found it fascinating how social movements often collapse when legitimized.

                                                              It’s like when a manager gives lip service to the concerns of an unhappy employee, making them feel like it’s all going to be better soon, but effort is not spent to actually change a situation.

                                                              When you walk around Google campuses, there is often material on the walls in common areas that talks about various social causes that Google is working to improve. It feels great to think that your organization is part of the solution.

                                                              The gap between superficial and structural control structures is interesting to pay attention to when seeking changes to a system. You can really dispel the risk of an insurrection by letting a Black Panther get elected, bringing in an external investigator to fix your sexual harassment problems, hosting a Noam Chomsky talk, etc… without risking any structural change.

                                                            2. 2

                                                              I think know what you’re trying to say, but if your opinion is “Google should stop existing” and then Google invites you to give a talk, what’s the point here? They’re not going to be persuaded into oblivion, so… why? As a pretense of open-mindedness? Or maybe it wants to be associated with the intellectual prestige of Chomsky? What’s the real reason, I wonder.

                                                              1. 3

                                                                An institution like Google might not even consider it to be at odds with a progressive, anti-capitalist view like Chomsky’s - it’s a different sphere with a different perception of reality. “Don’t be evil” is not just a empty phrase, these people really believe it. Moreover, the questions given by the interviewer where purely instrumentalist in nature: science is a tool for them, a means to an end. It’s an attempt to learn from an famous scientist, without considering the moral issues which are much more important to someone like Chomsky.

                                                                1. 1

                                                                  “Don’t be evil” is not just a empty phrase

                                                                  They dropped that a while back if you’re talking Google. The company has been practicing plenty of evil in surveillance sense, too. Hell, just the revenues versus spending on quick, security patches for Android by itself shows how evil they’ll be to their users to squeeze extra profit out. ;)

                                                                  1. 1

                                                                    I totally agree. What I meant was that the people behind the institution called Google most certainly have a different perception of evil.

                                                            3. 7

                                                              A friend’s workplace had protestors picket outside its door. The boss brought out coffee and donuts, warmly thanked them all for coming, and went back in. Within a half-hour, fed on the company’s dime and with no target for anger, they wandered off.

                                                              The Google employees watched a rousing argument from a famous voice. Really what they watched is their employer act totally unworried while a thousand other employees sat still. Next comes lunch or that mid-afternoon status meeting with the team in Australia. There’s no social movement started here. If Chomsky is lucky he planted a seed, but it’s pretty easy to forget someone ineffective telling you that you’re wasting your life and should do something uncomfortable.

                                                            1. 2

                                                              True. Maybe it should be called Reputation Engineering, part 2 or similar. Anyway, I eventually want to join the series into a single article so the titles will disappear.

                                                              1. 2

                                                                Also, note that if your title has “law” in it, it makes more sense to add the law tag. :)

                                                                1. 2

                                                                  Thanks, done.

                                                              1. 6

                                                                I like the idea of writing postmortems for abandoned projects. There may even be stadard POSTMORTEM file so that people would know where to find the cause of death.

                                                                1. 3

                                                                  “But that’s where the complexity kicks in: Oh! MSVC only works on Windows! Test X requires 8G of memory and the box Y only has 4G available. Shared libraries have .so extension on Linux, .dll extension on Windows and .dylib extension on OSX. I need to switch on valgrind for test X an box Y temporarily to debug a problem. Support for SPARC in our preferred version of LLVM doesn’t quite work yet. We need to use older version of LLVM on SPARC plarforms. And so on and so on.”

                                                                  This is exactly why logic programming was used in early expert systems for configuring things. It’s easy to express such things with declarative programming with programmer feeding in environmental data and correct configuration coming out the other end. Similarly, one project in academia was rewriting “make” in Prolog for much cleaner operation. Biggest, commercial LISP’s always embed Prolog in them for such things so you write most of your app in preferred language then drop into declarative programming where it makes sense.

                                                                  Best tool and default for most such jobs. That said, Cartesian is interesting work. The examples look clean like with other declarative programming. Plus it’s unusual for me to see cartesian solution to problems like this. I wonder what the inspiration was to try that.

                                                                  1. 3

                                                                    Good observation. I would be a fan of doing this kind of work in Prolog myself, but lets’ be frank: Prolog is a hard sell to real-world developers.

                                                                    As for the motivation, it’s twofold.

                                                                    Firstly, I have to deal with complex configurations on daily basis and most of what’s out there is pretty ugly. Approach in Cartesian tries to solve the ugliness without resorting to exotic paradigms. It’s (I think) something that your average JavaScript developer would be comfortable with, yet powerful enough to describe really complex systems.

                                                                    Secondly, it’s a methodological experiment. I think it’s fair to say that inheritance has mostly failed us when we tried to model the complexity and irregularity of the real world. Therefore, this is a sketch of an alternative paradigm, one that uses cartesian products instead of inheritance. (But, as mentioned in the README, it can even be combined with classic inheritance.)