1. 14

    This is often undervalued, but shouldn’t be! Moore’s Law doesn’t apply to humans, and you can’t effectively or cost efficiently scale up by throwing more bodies at a project. Python is one of the best languages (and ecosystems!) that make the development experience fun, high quality, and very efficient.

    As a Python programmer, this is a perspective that has never entirely made sense to me. Well, I should say hasn’t made sense to me for the last few years, at least. I feel like many people have this held-over dichotomy in their heads where Python is expressive and enjoyable, and thus one can write production code quickly, whereas other languages are not expressive and not enjoyable and thus code takes a long time to write. But while this might have been true in the past—while your performant options in the past might have all been some variation on fighting with the compiler, diagnosing obscure compilation errors, waiting for interminable builds—none of those are actually hallmarks of development in a typed, performant language anymore (except for C++). But modern compilers are fast, languages like Nim and D and Haskell are expressive and have powerful type inference. And generally speaking we are now in an era where a type system is not just a necessary evil for a compiler that’s too stupid to know how to interpret any variable without being explicitly told; they are universally recognized to be programmer aids, helping in writing correct code as well as performance. Without wading into the types vs tests debate, at the very least there is one—at the very least there’s a recognition that type systems, too, are for making the devlopment experience high quality and very efficient.

    If I were being cynical I would say that sometimes arguments like this feel like it’s really mostly about the “fun” part. That “programmer happiness” part, which is often conflated with programmer efficiency and expressiveness, but isn’t actually the same. It can almost feel like a hostage job—“I better enjoy the language I’m writing in, otherwise I couldn’t possibly be productive in it!”

    1. 8

      I find typed/compiled languages more fun actually, even C++. Because it drives me absolutely fucking bonkers to run a program and get a really stupid type error, fix, re-run, and get another type error. The compiler just tells you all the type/syntax problems up front and you can fix all of them with minimal rage.

      1. 6

        yeah, mypy and typescript have been a boon to productivity. Especially strict null checks.

        The advantages of the weaker languages is not having to play the “I have to make containers for all my thingy” games. Sometimes just a tuple is nice.

        Some of the existing typed languages don’t always follow the “if it’s conceptually simple, or if it’s easy for a computer to do, it should be simple in practice” rule. Especially when you’re crossing library boundaries and now spending a bunch of time marshalling/unmarshalling (sometimes necessary of course!) functionally equivalent stuff.

        Devil in the details of course

      2. 6

        I think your confidence in compilers is perhaps misplaced. It’s not just a matter of speed–other factors, like memory usage and even ability to compile period are relevant.

        none of those are actually hallmarks of development in a typed, performant language anymore (except for C++).

        I’d argue that the only widely-used performant typed language is C++, possibly Fortran (thought rust is getting close).

        The reason for this is that the farther you get into the problem domain (and the more comfortable it is for you), the farther you move away from actual silicon running instructions. It’s not a false dichotomy.

        The best-performing code will be written in assembly, but it’ll be terrible to deal with as a human (because we aren’t computers). The most comfortable code will be written in a high-level language (ultimately taken to extreme of “hey, grad student, write me a program to do X”), which is exactly not what runs on silicon.

        1. 4

          I think your confidence in compilers is perhaps misplaced.

          Now include python on the same plot, and the axes will stretch so far that GHC will look indistinguishable from GCC.

          the farther you get into the problem domain (and the more comfortable it is for you), the farther you move away from actual silicon running instructions. It’s not a false dichotomy.

          It’s only a true dichotomy if the human is better at telling the silicon how to implement the problem than the compiler is, which gets less true every day. It’s already the case that GCC will often beat hand-coded assembly when trying to solve the same problem. And my experience is that on real business-sized problems with ordinary levels of programmer skill and limited time available to produce an optimised solution, Haskell will often comfortably outperform C++.

          The best-performing code will be written in assembly, but it’ll be terrible to deal with as a human (because we aren’t computers).

          These days assembly is a long way away from reflecting what the actual silicon does. To first order the only thing that matters for performance these days is how well you’re using the cache hierarchy, and that’s not visible in assembly code; minor tweaks to your assembly can lead to radically different performance characteristics.

      1. 7

        The author recommends learning about session management before digging into the window management side of things, which I think is reasonable. One software suite that makes that a little easier is abduco + dvtm, which are designed to work together, but don’t need to be, and separately handle the session management and multiplexing (respectively) that tmux/screen couple together. I am not an avid user because I haven’t found the great need for any of the above quite yet, but I found them very helpful in demystifying what tmux is supposed to be so great for.

        1. 1

          Thanks for the links. Factoring tmux out into abduco/dvtm does seem conceptually nice, and clarifies the building blocks of what tmux is doing. Although if I were to always use them as a pair I’m not sure there’d be a big practical advantage to them being separate binaries. Seems like the advantage would come if I were to sometimes use just one or the other on its own.

        1. 3

          I’m not giving up colorful text schemes or immersive experiences anytime soon, but one nice affordance of the acme scheme the author uses is that it could make coding on low-color outdoor displays (e-ink, transflective lcd, epaper) more usable.

          1. 3

            Yes, it got simpler to use projectors too!

          1. 4

            This is off-topic, but do you live in Sunset Park? I live here too and was surprised by the blog name.

            Anyway, great post. It’s going to take me a few reads to process but there’s a lot of interesting stuff here. Thanks.

            1. 4

              I used to :) Now I live in Windsor Terrace. You know, I have to rep BK.

              1. 2

                South(ish) BK represent!

            1. 16

              I have fought against this myself. It’s hard. What I found really helpful was to decouple my self-worth from my job. Nothing related to programming skill is related to peoples’ inherent value; almost nothing in programming is a moral decision (except being willing to say “this is a job I will not do”). Personally I have found it easier to find this from philosophical, theological, and moral texts than from self-help books; tastes vary.

              If you are interested, I recommend these books:

              • The Miracle of Mindfulness by Thich Nhat Hanh
              • Xunzi (translated into English by Hutton)
              • Meditations by Marcus Aurelius
              • The Confessions by St. Augustine
              • The Seven-Storey Mountain by Thomas Merton

              Several of them are “religious” (Mindfulness is Zen Buddhist and Augustine and Merton are both Catholic), but there is a common thread of self-critique and examination that runs through them that I found really valuable.

              1. 8

                What I found really helpful was to decouple my self-worth from my job

                I will go one further, and say what really works for me (and what I’m constantly having to practice) is to decouple my self-worth from my own intelligence or talent, and to go on to admit that whatever I happen to be suited to, there’s very little credit I can take for it. As a programmer, I am entirely reliant on prostheses: documentation, yes, but also unit tests, type systems, and mathematics—these are all useful to me, and everyone else, specifically because they help fill in the gaps where my reasoning ability (supposedly the thing I am proud of as a programmer) is deficient.

                Nevertheless my mind does rush to judgment constantly; it’s been fine-tuned to always find a way to set myself apart from whoever I’m looking at. At then end of the day, when it comes to this profession, pretty much every single one of us would benefit from approaching it with a huge degree of humility.

                1. 11

                  Nevertheless my mind does rush to judgment constantly

                  Programming encourages this, because code has to be right. Pointing out mistakes is something we frequently have to do as a result.

                  Most of ‘the real world’ doesn’t need the same kind of correctness. Businesses run on approximation and best efforts.

                  This is (to my mind) the most significant ‘Déformation professionnelle’ of the programmer. In the rest of society, maintaining a relationship is (often) more important than pointing out a mistake.

                  1. 4

                    …every single one of us would benefit from approaching it with a huge degree of humility.

                    This is the key point, I think. Humility helps you see where you’ve made a mistake; where you can improve; where you might be entering an area of weakness. It also helps you relate to your coworkers and colleagues. If you can come to recognize things as opinions, rather than subjects of Objective Truth, that don’t really matter very much, then a lot of friction is removed.

                    1. 4

                      Humility for me is really hard when met by arrogance. Arrogance provokes arrogance in myself.

                      1. 2

                        That is very true. It’s easy to get offended and act arrogant/negative in return. This is probably my primary failure mode! Still, it’s just something to recognize and work on.

                  2. 2

                    Thanks for your comment, especially the small sentence “it’s hard.”

                    I struggle with the concept of decoupling my self-worth from work. Sometimes, I think that this is the right path. Then I cannot perceive how something on that I spent so much deliberate time & energy should be irrelevant for defining my self? I am currently digging into mediation, maybe your recommendations provide further guidance. Thanks.

                    1. 4

                      Consider these ideas.

                      1. Imagine that there is an economic down-turn and you are not able to keep your job. You are forced to work, to make ends meet, as a cook in a restaurant.
                      2. Imagine that you were struck by a car while crossing the street. You have a head injury and, while you are able to walk and talk, are never able to work as a programmer again.

                      In either case, should your self-worth be damaged? I would say no. How we treat our personal obligations is what defines us as people, not our work. If you are meeting your personal obligations as best you can in the circumstances—treating the people around you well, taking care of your children, generally making the world better—then you’re doing fine.

                      I really like Xunzi for this, because he sets out his goals plainly.

                      The gentleman is the opposite of the petty man. If the gentleman is great-hearted [confident] then he reveres Heaven and follows the Way [i.e. follows social rituals and educates others]. If he is small-hearted [shy] then he cautiously adheres to yi [moral standards] and regulates himself. If he is smart, then with enlightened comprehension he acts according to the proper categories of things. If he is unlearned, then with scrupulous honesty he follows the proper model. If he is heeded, then he is reverent and reserved. If he is disregarded, then he is respectful and controlled. If he is happy then he is harmonious and well-ordered. If he is troubled, then he is calm and well-ordered. If he is successful, then he is refined and enlightened. If he is unsuccessful, then he is restrained and circumspect.

                      In short, there is a way to be a “gentleman” (or “sage”) in every circumstance. Of course, nobody is perfect, so it’s better to be seen as a “process” than an end point.

                  1. 12

                    I like that the next comment is “throw it all away and start over”. Never could have predicted that…

                    But otherwise, I have no idea what this is about. Random comments on random github issues don’t convey much information without a lot of context digging.

                    1. 26

                      Crystal is strongly typed, but overwhelmingly uses type inference, rather than explicit types. Because Crystal aims to be spiritually—and frequently literally—compatible with Ruby, that’s a problem: to accomplish that, Crystal relies on sometimes-nullable types with implicit structure and implicit unions, such that, frequently, the only way to even begin type inference is to load the entire program’s AST into RAM all at once and then start your massive type inference pass. What you’re seeing in this thread is how a “simple” fix to a YAML parser error reporting hit that problem, causing Crystal to use a critical amount too much RAM and OOM.

                      I think there’s probably an interesting discussion here about how a language that relies on implicit strict typing really needs a carefully thought-out type system (or language compromises, such as OCaml’s .mli files) to be scalable. But I agree with you that it’s hard to have that discussion based on this GitHub issue thread.

                      1. 3

                        Thank you for this explanation! Apologies that I posted it without context. May I quote your comment in the post text?

                        1. 4

                          Sure.

                      2. 5

                        I agree with you about linking to a github comment being a poor post. I had to read through more comments, as far as I could gather Crystal uses an obscene amount of memory when compiling so you cannot make anything large with it. But hey, maybe that’s actually a good strategy to keep things smaller.

                      1. 3

                        issues I have with productivity, focus, and when it’s getting hard to sleep, and everything’s getting to me, and I’m coming into the office around noon, my tardiness

                        The great news here is that you have a pretty concrete list of patterns of behavior that are holding you back—and “acumen” isn’t one of them. So you can feel pretty confident that you’ve got the raw materials.

                        I was in a very similar place, actually. I started programming a long time ago, loved it, but failed out of school; and failed in my jobs after school for a lot of the same reasons you mention. I was smart and had a natural talent for programming, but I also:

                        • was also majorly depressed
                        • didn’t know how to motivate myself beyond my own intellectual interest (which was never omnipresent)
                        • dealt with stress by procrastinating and staying up late
                        • which meant I was often fatigued and had less ability to focus.

                        As it happens, I did get a simpler, lower-paying job, which I did for 7 years. It was never very satisfying. But it did give me enough time to teach myself a lot of the mental habits necessary to resolve the laundry list of issues above. In particular I:

                        • developed a pretty solid meditation practice
                        • built a series of strong support relationships
                        • started going to therapy.

                        By the time I realized I didn’t actually want to do what I was doing (I had assumed I was actually a terrible programmer since I failed the first time), I was in a place to keep myself grounded and deal with the stress of working at a higher level. I needed a lot of work before I could bring to bear my natural talent on a practice that I really loved.

                        I don’t know you, but the things you mention up there feel like similar issues to me. They sound like the patterns of a mind that probably struggles with some kind of depression, and can get into spirals of aversion and withdrawing when it encounters stress. I could be totally wrong, of course. But I bet that some of the habits you’re describing can be reformed. The benefits to this kind of practice are great.

                        1. 3

                          I’ve been thinking a lot recently about how much coding is done without thinking. Just throw stuff at the compiler until it seems to work. In Pragamtic Programmer, they call this programming by coincidence. I find myself doing this out of laziness, and it’s not good. It’s been a while since I’ve written anything by hand, but I’m pretty if I tried it would result in better code. I’d be wary of anybody saying they can’t program by hand.

                          1. 2

                            That’s funny, because at the end of your first sentence, I was agreeing with you, on how all the real programming I do is on an intuitionistic level, below critical thinking.

                          1. 3

                            Can anyone familiar with Nim comment on the quality of the contents? I like the “in Action” part as it shows real examples but would like to learn idiomatic language if possible.

                            1. 4

                              Author here. Unfortunately I cannot comment on the quality as I am biased. But the book does show many practical examples and thus teaches you the idiomatic way to do things. Hope that helps :)

                              1. 4

                                dom is being a bit modest below - in addition to being the author of the book, he’s also the second-most prolific contributor to the language itself and has written much of the core tooling besides. Which is to say—you can trust that you’re learning idiomatic language from him because he wrote the idiom.

                                1. 2

                                  Excellent reply! That’s what I wanted to hear. Thank you.

                              1. 24

                                itamar, you’re sure to be pilloried by every ad/fin/etctech engineer who has been assuaging the niggling and constant awareness that they’re not actually producing anything of intrinsic value with the banality that they’re working on “interesting problems”. But you’re right on the money - it’s not that they need to meet your, or my, standard of usefulness; but they still have to meet their own. We can all smirk about the excesses of SV and the money that’s thrown around, but we’re the ones making things that do something. Or don’t do something. Each of us as individuals—and all of us as a trade community—need to fully reckon with how we are affecting the world every day. Nothing happens in a vacuum. Not even Snapchat.

                                1. 3

                                  I can only imagine it’s because in most places, the number one reason to hire a new developer is because you want to increase your bandwidth in terms of LoC. The prospect of hiring a developer not to write as much of the code you need the most right now runs counter to that.

                                  1. 25

                                    What a mature and gracious signoff. A model to be emulated.

                                    1. 1

                                      It turns out that in D, if you declare an enum with only one value (in this case a list), it is declared to be a manifest constant. That seems to me like a slightly strange way of getting a compile-time-only value, but I’m not a D programmer.

                                      Actually, I suspect it’s “you’re not a compiler writer who wrangles DWARF debug output format for a living.”

                                      In which case DW_TAG_enumerator is the only / obvious applicable hammer for that particular nail.

                                      http://dwarfstd.org/Download.php

                                      Alas, a language designer has a hard choice…. to design a language and write a compiler……

                                      Or design a language and executable format and a debugging information format … and write a compiler and a linker and a debugger and a loader and a way of linking to pre-existing elf code and debugging that…….

                                      Walter Bright and company has wisely opted for the former path.

                                      However I suspect it does, shall we say, ahh, “inform” certain choices.

                                      1. 1

                                        I don’t follow… Can you break it down for me?

                                        1. 1

                                          EDIT: Disclaimer: This is not a D official view… This is a wild arsed guess based on way too much time staring at objdump -S and objdump –dwarf=info for various C / C++ programs.

                                          I think the,

                                          • from the C programmers perspective,
                                          • strange choice of the enum keyword for this use case….
                                          • probably relates to the fact it uses exactly the underlying mechanics
                                          • that the compiler backend / linker / dwarf debugger format… uses for enum values.

                                          It may be tempting to create a new category for these things, but unless you rewrite the rest of the toolchain…. you’re pretty much stuck with being creative with whatever is already there.

                                          Even C repurposes the linker mechanics that was set up for ye olde FORTRAN common blocks, and C++ does horrible post processing on .o files to set up static constructors.

                                          So I think this is a case of a compiler writers perspective leaking into the language design.

                                          If you look really really closely at C constants… and what happens at the assembler / linker / debugger level, they are a bit… ahhh. Strange. Not quite what you’d think on first glance what they mean. And the debugger support for them is ahh, stranger. And there are subtle differences between what C++ and C means as well.

                                      1. 4

                                        The link points to a download page for an html file.

                                        1. 1

                                          Strange, they must have their MIME types misconfigured on the server - I just sent an e-mail.

                                          http://peters.gormand.com.au/Home/tools is a link that seems to work correctly.

                                          Also, the whitepaper is old (1998) but still valid: https://bit.ly/2rN3ycg

                                        1. 3

                                          Why do folks have such a hard time looking for 3rd party packages? An immutable, sorted map doesn’t come up all that often (I don’t think I’ve ever “needed” the immutability bit), but there are packages out there that can do it. For example: https://github.com/Workiva/go-datastructures#dtrie. *

                                          It’s not type safe, but it fits the requirements he wanted.

                                          • caution: I’ve never actually used this package
                                          1. 4

                                            One of the chief dangers of excessively verbose and inflexible code is not just that its implementer has to do a lot of typing; it’s that all that typing provides a high surface area for bugs and generally difficult-to-reason-about implementations. This sort of issue affects whoever has to use the library, not just whoever has to do the implementing.

                                            It’s also, by the way, generally true that code which somebody else wrote is going to be more general and, you know, not written by you, and therefore magnify the verbosity and difficult-to-reason-about issues.

                                            1. 5

                                              Why do folks have such a hard time looking for 3rd party packages?

                                              Sometimes people don’t want to add external dependencies for things that, in some cases, are (relatively speaking) straightforward to implement.

                                            1. 3

                                              I’m in favor. Indirectly I work with fortran every day; it’s still at the heart of the biggest matrix operation libraries in the form of blas/lapack.

                                              1. 6

                                                Point of fact: I have no idea what a PCI bus is.

                                                1. 9

                                                  PCI is the bus that personal computers and servers use to communicate with peripherals. Your USB host adapter (the thing that connects to your USB peripherals), network card (ethernet or WiFi), or GPU are often connected to the processor via this bus.

                                                  If I recall, much of the PCI peripherals appear as memory-mapped I/O to the processor. So in the OS/driver software, you can trigger the interactions with the device by reading/writing to a buffer in memory and then setting control bits on the device to go read/write from that buffer. The device can trigger interrupts on the processor to notify it that an operation is starting/completing.

                                                  … lots of other details missing depending on how shallow/deep you want to go …

                                                  1. 7

                                                    Did you ever own a desktop computer or see inside them? Notice you plugged the network, graphics, USB, etc cards into slots? Those slots are connectors for PCI that let your devices talk to the CPU and memory. One important thing about it is it’s ridiculously fast and low latency compared to other buses. It’s so fast that I once thought about using it directly as a network switch for high-performance computing (i.e. clusters). Turned out companies already did with lovely numbers:

                                                    https://www.enterprisetech.com/2014/03/13/pci-express-switching-takes-ethernet-infiniband/

                                                    Dolphin’s stuff I Googled real quick claims up to 64Gbps per port. High-speed PCI’s are awesome. :)

                                                  1. 2

                                                    There does seem to be an interesting trend (I hesitate to say “sea change”, much as I might like to) going on here. It might just be where my head is at right now, but it doesn’t seem like I’m the only web programmer, who came up in a dynamic, interpreted language (for me it’s Python), now turning back to the world of static typing and compilation in order to reap some of the benefits of greater type safety and greater performance.

                                                    It might be a matter of the web stack wave cresting a little bit; Django and Rails are quite well established and the pattern they provide of quickly spinning up a serviceable web app is no longer a novelty; it’s an expectation.

                                                    For my own part I know it’s also a matter of my professional needs naturally expanding into areas where performance is a real concern and Python just can’t keep up.

                                                    But it seems like it’s also a matter of this new generation of performance-oriented languages that have really learned some of the lessons around expressiveness that Python and Ruby have taught us. Having been educated in C++ and a little bit of Java (which I particularly hated writing), the expressiveness and ergonomics of Python were an incredible breath of fresh air. When it came time to start writing more performant code, I was frankly not particularly willing to dip back down into C++, with all the syntactical unpleasantness and memory management that entailed. But we no longer have to make that tradeoff; Go, Rust, Nim, Crystal (and more to come—Myrddin next?) all represent a healthy new crop of languages that have decided to return to bare metal, but are willing to adopt many of the programmer-friendly advancements of the last 20 years or so.

                                                    1. 4

                                                      I gotta say, this would make me awfully nervous. I tend to agree with the dictum that you should be including at most one new and sexy technology in your stack at any point. The hype on Elixir and Elm is super high right now—that’s not to say that either one of them is overhyped, or undeserving of its attention, but if I read about starting one new project where essentially the entire stack for both the front and backend is brand new, I’m going to start wondering whether these decisions were made for engineering reasons or because the author wanted a chance to play with some new and exciting tools.

                                                      1. 4

                                                        I’d put Elm in the “new and hip tech” bucket but you should remember that Elixir is based on decades-old Erlang/OTP experience. If you really want to reduce Elixir to a short sentence, it’s cleaned up, syntactic sugar over Erlang with macros (among other goodies).

                                                        Elixir is hyped, sure, but at least its claims about fault tolerance and distribution have historical proof because they’re Erlang’s claims.

                                                        Something that stands out to me about the Elixir/Elm pairings is that people focus on Elm’s types as a major point yet use a dynamic language for the backend.

                                                        1. 4

                                                          I’d put Elm in the “new and hip tech” bucket

                                                          I would argue that while Elm is still currently in that bucket, it is start to leave. The language is becoming more stable on the whole, and the ideas it promotes have already spread to almost every other frontend framework available. If you’re writing Redux, you’re writing Elm without types. With regards to the types, Elm uses a basic type system which has existed in MLs for decades. At this point, there is little that Elm introduces as hip - instead, it is more the coagulation of a few very popular ideas elsewhere. Elm’s type system has historical proof because of this. The Elm architecture has been popular in concurrency groups for a long time, too.

                                                          Something that stands out to me about the Elixir/Elm pairings is that people focus on Elm’s types as a major point yet use a dynamic language for the backend.

                                                          Agreed. Elixir’s typespecs just don’t match up with the types-as-a-feature of Elm. To me, the biggest selling point of Elixir is the build tooling that helps simplify getting started with Erlang. Rebar is dead, long live rebar. Honestly though, if I got to choose the backend for a new production job, I would probably choose Haskell. I’d rather have a proper type system than not. This is not necessarily true for other Elm users - I see a lot of them coming from Rails or JS backgrounds, and Elm’s type system is often the first proper type system they’ve used.

                                                          1. 1

                                                            Fair enough, I do think Elm is a safe choice nowadays (I have a friend who works at NRI ). Like @zdsmith I do see it “hyped up” a lot so I got the wrong impression. Thanks for the explanation.

                                                        2. 3

                                                          Usually, I’d agree with you. Especially when looking at the JS world. But many of the interesting concepts in Elm and Elixir have one significant advantage over “hip.js”: It’s very hard to screw up your code. The lack of this feature in Coffeescript and ES7, for example, makes it easy to misinterpret or half-heartedly apply solutions one doesn’t understand, making it hard as hell to maintain the software afterwards.

                                                          But proper typing, pure functions, good error messages, and time travelling debuggers are all things that are hard to misuse and generally lead to better code. These languages teach better programming, IMO.

                                                          1. 0

                                                            whether these decisions were made for engineering reasons or because the author wanted a chance to play with some new and exciting tools.

                                                            This is basically a mass-hallucination in the Elixir community. For some reason the Elm meme has become deeply embedded in the brain stem of some Elixir devs, not for any good reason it would seem.

                                                          1. 13

                                                            I think both the original article as well as this followup miss the point that type systems aren’t, entirely, about preventing errors- at least not in the same way that tests are about finding them. Tests should be about checking assumptions, making sure that what you wrote matches up with what you intended to write, and provide a mechanism to check that your assumptions still hold under changes to other parts of the system.

                                                            In my opinion the value of tests is that they provide an abstraction over the type of data you can be working with. Sure, the type checker will provide errors when you make a mistake, but a large part of the value is simply in allowing or forcing you to think about your data algebraically or categorically. The benefit to code quality is that people make fewer mistakes when they have more power and flexibility for how they are expressing the code. When people are complaining about the difficulty of working with strong statically typed languages it seems that it’s often because they haven’t learned to exercise that style of thinking to the point where they see it’s benefits to expressiveness.

                                                            1. 4

                                                              I’m interested in this, but I’m not sure I entirely follow. I get:

                                                              1. Type systems aren’t (entirely) about preventing errors.
                                                              2. Tests are about finding errors, checking assumptions, maintaining invariants.
                                                              3. Part of the value of types is allowing you to think algebraically/categorically.

                                                              I think that (3) is a follow-on from (1). Can you elaborate? I’d love to hear more.

                                                              1. 6

                                                                I’m actually in the early stages of writing a book about this; I’ll try to distill my thoughts down to something makes a little more sense here. For a bit of perspective, the languages I’ve worked with most in my career are: C, Haskell, and Ruby. Much of my thinking has evolved out of comparing the general approaches that I’ve used in those three languages, and trying to generalize them to the wider practice of programming.

                                                                To start at a very high level, I think of the Curry-Howard Isomorphism, which basically says that a type signature for a program is a formula that is proved by it’s implementation. In that sense, when we’re working with a strongly typed language, our entire program is a composition of many tiny proofs that we develop with the help of our theorem prover (in our case, the type checker). If take a category theoretical approach to program analysis we can see that each of these proofs can have several different implementations that may or may not be equivalent for our purposes. For example (*) and (+) both prove (Int -> Int -> Int) but we can’t use them interchangeably in our application.

                                                                When we start looking at developing applications this way, it leads us to consider an approach where we start with some set of formulae that we want to implement as programs, which are themselves pure functions. Our tests then are acting much more like function-level acceptance tests than unit tests, and our focus is not on the proof of our formula but rather on demonstrating that we have selected the correct morphism.

                                                                Moving back into the “Not about preventing errors” part of this though, taking a category-theoretical approach to software design also means that, beyond the area of proofs for our applications, we have an entirely new toolbox to use when we’re thinking about how we design our applications. I typically use the language of algebra instead of category theory because it’s often sufficient and less intimidating. If we start to look at our application as a type level formula then the process of proving it with our program becomes one of defining an algebraic structure that’s appropriate for our data domain, and then building up the set of morphisms we have over the objects in that structure, and finally using the algebra we’ve defined to prove our formula.

                                                                I believe that the benefits of this approach extend beyond making our code less error prone, and improve the overall quality of our code under refactoring, the addition of new features, or changes to requirements. While traditional approaches to unit testing will give you the confidence to make changes without introducing regressions, the type-driving algebraic approach gives you a coherent language to work with and I think makes it much easier to write applications that are structured such that they facilitate modularity and refactoring. When you’re building an algebra it’s difficult NOT to build small orthogonal components that can be composed into different programs as your business requirements grow and change.

                                                                1. 2

                                                                  Thank you very much!

                                                                  I can’t claim to fully appreciate your insight into types… I’ve tackled the Curry-Howard Isomorphism and Category Theory many times now, and every time my brain just fuzzes and I only get the most general gist. So I’m still waiting for the breakthrough that will really allow me to apply that whole side of things to programming. Maybe your book will be the thing!