I don’t mean to post this to trigger a religious war. I think it’s an interesting discussion. I’m a big Python programmer, but I also love Go. I see both sides of the discussion and I think this article does present a useful take on it.
I don’t mean to post this to trigger a religious war. I think it’s an interesting discussion. I’m a big Python programmer, but I also love Go. I see both sides of the discussion and I think this article does present a useful take on it.
The best data point I have on this personally is my current project/team. The codebase is over 500KLOC now. The majority of it is in Python, followed by JS. I’ve been working on it since the beginning—over 4 years. We’ve built components and then extended them way beyond their original design goals. There’s a lot of technical debt. Some of it we’ve paid down through refactoring. Other parts we’ve rewritten.
As time has gone on, we’ve gained a better understanding of the problem domain. The architecture of the software system we want is very different than what we have or what we started with. Now we’re spending our time figuring out how to get from where we are to where we want to be without having to rewrite everything from scratch.
I agree we have the lava layers problem the author describes, with multiple APIs to do the same thing. But I’m not sure if we would spend our time unifying them if we had some kind of miraculous tooling afforded by static types.
Our time is better spent reevaluating our architecture and enabling new use-cases. For example, one change we’ve been working towards reduces the turn-around time for a particular data analysis pipeline from 30 minutes to 1 millisecond (6 orders of magnitude). Now our product will be able to do a whole bunch of new cool stuff that was impossible before. It took a lot of prototyping to get here. I don’t think static types would have helped.
My team’s biggest problem has always been answering the question: “How do we stay in business?” We need to optimize for existence. We’ve had to adapt our system to enable product changes that make our users happy. Maybe once your product definition is so stable, like Google Search or Facebook Timeline, you can focus on a codebase that scales to 10,000 engineers and 10+ years of longevity. I’ve not worked on such a project in my career. The requirements are always changing.
I do not know what this exactly looks like within your team, but in other teams I’ve seen optimize for this it becomes very dangerous. In those teams, the problem is it’s a fear-based metric that is almost entirely unquantifiable unless you really understand the business, and even then it’s hard. In the face of a perceived existential threat, that threat will always win compared to doing something good. It is not a question I would ever propose a team use to make technical decisions.
For my own experience, my most recent project has been about 3 years in a dynamically typed language. The system is not so large in the language (Erlang) but that is because Erlang is pretty expressive, and we aggressively unified concepts that weren’t business logic. Turns out a lot of systems only really need a few concepts and by making them composable we built new features out by composing them in different orders with some business logic at the end.
In our case, a statically typed language would have been nice because the we had to enforce so much discipline by convention, which can be exhausting after enough code reviews. After long enough time, people on the team converge on that discipline but any time someone new is added to the team it happens all over again. Most of the discipline we enforced was just good practice in a statically typed language. One particular sore point was a central framework that someone in the very early stages wrote that was absolutely painful to understand and unfortunately leaky enough that you needed to read it every now and then. This was eventually refactored out just for our own sanity but it was a long and very painful process because we had no types guiding us. But, I’m a statically typed fanboi so I will view most problems in terms of if the type system will help me with.
Lava layers. Love it. Thank you for that.
It gets really fun when you rename the old API to oldapi. But then you already have one, so now it’s oldoldapi.
Anyway, I was wondering if maybe dynamic languages can’t help here. The flexible typing means you can change the existing API. I’ve seen some projects get stuck with old Apis (especially in C) where updating code to refactor is considered too much work. Some amount of polymorphism and introspection helps a single API actually be all the Apis. There are static languages with those features of course, though dynamic ones seem to have better support in general. Not sure if this is a good idea, it probably trends towards a ball of mud.
Just the first thought I had after reading static typing lets you change everything is that static typing requires you to change everything. I would say that’s a good thing, but it’s also pretty discouraging when you’re drawing up plans.
My experience is that’s where you end up with the worst problems. Say your
renderUserPagefunction falls back touser.firstNameanduser.lastNameifuser.namedoesn’t exist, because the v1 user object had those fields instead. No-one ever knows when it’s time to remove that code - maybe no v1 user objects have been created for the past 5 years, but maybe there’s still one knocking around in that API that’s only used by the JavaME app. So you end up with these “flexible” APIs that just stay there forever and are impossible to maintain.Better support in what sense?
Typeclasses seem like the right way to solve this. I’d create a
NameRenderer[oldapi.User]instance, require[U: NameRenderable]in therenderPagemethod and ripple it up the tree. Later on I can just delete the instance and see whether I get a compile error or not.As a determined static-types advocate, I suppose I should weigh in here. :)
Briefly, this benefit hadn’t occurred to me in nearly this level of detail. While I’ve worked with large Haskell codebases, there are none I’ve continuously used for ten years. It’s very much the case that, when the type system can express your invariants, the cost of change is much lower even in terms of finding everywhere you need to change, and an even better deal when you don’t have to update a test case as well.
Of course, test cases are often needed in addition to type signatures; however much I dream of dependently-typed languages checking all their invariants statically, large systems built in that fashion are still a few years off.
Regardless, it’s really nice to have this perspective, especially given that it’s based on direct experience with the same codebase over that length of time.
Edit: s/major/determined/ because I’m not famous and it was bugging me to appear to claim otherwise. :)
After getting used to Haskell, refactoring in Ruby felt nearly impossible. You could do it, and we did do it, but you spend the whole time wanting to scream. Massive amounts of unit testing is needed and usually—not that this is best practice, but it is an easy practice to fall in to—these tests are hideously overlapping and complex since refactoring tests is the same game as refactoring code… just often without tests.
So each time you make a change you can anticipate a screen full of failed tests which have grown like scar tissue over the affected area. This makes even the worst Haskell compile-time error message cute in comparison. Many of these tests are actually now out-dated—the right solution is to excise the test. Many others are long range cascading effects allowed by “duck typed” interfaces which let bad tags penetrate deeply into code. A few of them, if you’re lucky, point near to the place where the right change is needed. You spend the whole time trying to decide between continuing your intricate surgery or just slicing away all the necrotic bits and going in fresh.
Haskell refactoring can get hairy, but usually only when you’re doing an admirably hairy thing yourself. You get a sense for the places where types do not exist and you build some fear for them. You’re going to need to fall back on testing at this point and that’s going to take waaay longer to handle than a mere recompilation loop. It’s also up to you to, in the worst case, build example cases and hope that they’ll stay relevant for a while. At least with QuickCheck you can stay at the level of properties your code is supposed to have.
I’m all for the argument that dynamic languages have fast startup times and sometimes that’s what you need to optimize for. While I don’t believe that static languages lack fast startup times in some ways, it’s absolutely undeniable that the culture of “fast startup” which has defined communities like that surrounding Rails is impressive and an investment of a hundreds of man-decades to replicate. If that’s exactly the kind of leverage you need then it’s hard to walk away from it. But there are other options out there.
Your comment about fast startup time has put the image of the Ariel Atom in my mind. In BBC’s Top Gear, they test a car against a motor bike, and although at the offset the bike is faster, when the first curve comes, the car whizzes by and never looks back: its design allows it to stay more stable when you need to change directions. I may be looking to much into this, but I see an analogy to typing disciplines: a dynamically-typed language can help start off fast, but once the first big refactoring comes, the statically-typed language would have allowed for faster turn-around time.
My experience corroborates this. I started out in PHP land before moving to Perl, then to Objective-C with its optional typing, then Java and Scala, while now looking at Clojure.
For me the difficulty of refactoring was a major driver in moving to languages with ever more strict typing. But I have found I much prefer writing code in dynamic languages. Which brings me to Clojure… one of my favourite features of Scala turned out not to be its strict typing but increased focus on immutability—and Clojure takes this even further. Although I haven’t used it yet, I am hopeful that Clojure’s optional typing (
core.typed) can help Clojure provide the best of both worlds: rapid and convenient initial development, with added on type safety to aid refactoring as the code matures and requirements change.I didn’t find core.typed to be sufficient, so I moved onto Haskell after Clojure. YMMV.
Explanation here
Immutability alone doesn’t really do that much, particularly when side effects and mutation can happen anywhere in a Clojure program and you can’t even use types to prevent as much. I get a canary in the type (IO or ST) when effects or mutation could happen.
Very similar experience here as well. Just recently wrote a post about that for the Commercial Haskell group: http://www.kurilin.net/post/117369543198/haskell-at-front-row