This is my 1,000th Lobsters story.
When you’re building a doghouse and you’ve only ever built doghouses and all your friends build doghouses and the other houses you see look like doghouses, all that skyscraper stuff is bullshit.
My favorite quote out of the bunch. I’m waiting for you to write an article titled “Functional Programming: Good News, Bad News, How to Win Big”.
I think the “types are tests” idea is great but only goes so far. I’m recently in love with a Conor McBride quote that goes something like: “getting types right changes gravity to pull you toward the right answer”.
What’s different? Types are “half executable”. They set of some component of the framework of your actual code instead of sitting alongside like tests do. They are also constructed in a logical language so that they are internally consistent in a way that snowballs instead of growing only (sub) linearly.
This means that you end up writing types as a shortcut to writing good code. They’re more like the skeleton in your body—living, internal—than the scaffold of a skyscraper.
That’s a really nice extension of the idea, thank you. Nice meeting you at Lambda Jam, I hope this essay makes more sense than the jumble it came out as when we chatted. :)
It was great meeting you, too! It’s very nice to see these ideas grow!
This is so important. We are still building web (and other) software like amateurs, and calling ourselves professionals because of our big tools. There’s a common idea that once you get the right framework/library/text editor/OS things will get easy. In fact, when complexity increases, tools matter less, architecture and process get in the front.
Once the whole project gets more complex, you need more reliability. It is easy to debug 1000 lines of code. It is harder to debug multiple services with mutual dependencies when one of them does not check its invariants. It needs a systematic approach to software quality. Unit tests are useful, but clearly not enough: you need mutation testing to make sure the tests are relevant, and property based testing to verify larger ranges of values. Types are useful, but unless you’re using dependent typing, some business logic will slip through the cracks. Do you have continuous integration? Do you know the performance bounds of your system (not “I know it is fast” but “I know its steady state, and when it will break down”) ? How do you handle failure? How do you document failure?
That kind of testing takes time, but that’s how you can build rock solid components.
You mean the fact that my dev environment is Vim+GNU/Linux doesn’t automagically make me an epic developer? That makes me sad.
That is absolutely how you build rock solid components…
…that aren’t being sold, that took too long to develop, that lose their solidity once normal developers who don’t know a fucking thing about proper engineering infiltrate your project, etc. etc.
The thing is right now that 99.99% of people only want a doghouse, and even the skyscraper folks have only budgeted for a duplex.
Quality is not a goal unto itself.
I really agree and didn’t want to present quality as a goal in and of itself. I see skyscraper to doghouse as a spectrum, most problems as higher up that spectrum than we assume, and the cost of appropriately addressing them to be much lower than we fear.
Quality is not the goal of a project, it is a requirement for which you have to make compromises. Sure, for a fire and forget web app, you won’t need to care about reliability. But for a library used by somebody else, for a web framework that will be reused, for for any project that must be maintained long term, focusing on quality will make the difference.
The things I cited, mutation testing, property based testing, typing systems, performance testing, they are easy to do right now. And putting them through a CI (which is also rather easy these days) ensures the quality won’t drop in the future, even when you get new developers onboard.
The real problem when making those compromises is that we automatically assume skyscraper<->good quality, doghouse<->bad quality. Those are orthogonal dimensions: complexity and quality. There are complex projects badly handled, there are simple projects really well done. What people often settle for is to get lots of features (ie increased complexity) shipped fast, and this is the recipe for unmaintainable software.
We’ve built software as loose bags of features atop a teetering tower of unreliable abstractions. Functionality is slow, regressions are common, and that’s even before you throw in the now-standard business practice of shipping as part of figuring out what’s worth building and how to build it. Maintenance often means burning it down and rebuilding from scratch.
I feel like I can relate to this much more than I would like to admit.