1. 23
  1.  

  2. 17

    All in all I’m being much more careful when researching dependencies and libraries before bringing them in.

    I’ve recently started reading the packages I bring in more and its shocking how low the bar is sometimes, or how insanely abstract & non-idiomatic many of these libraries are, or worst of all a small error-prone wrapper of another library. Often I find I don’t even need to pull it in, there’s the 5-20 important lines of code I really care about & I can copy it in - at least that way I can understand & modify it.

    This wasn’t the example in the article but in my experience NPM seems to follow a variant of unix philosophy, “do 1 thing”, but miss the “do it well”. I found another package we used that wrapped puppeteer to do very simple preset options for it, we had an issue with it & we couldn’t manually upgrade puppeteer or change what is passed to it because this in-between library decided it knew what we wanted.

    One funny thing I’ve noticed, I get a lot of push back for copying in some (unimportant & infrequently modified) code in that is slightly obtuse, but rarely has anyone ever batted an eye when I add a dependency.

    1. 4

      I had a similar experience, and it made me write a blog post: “never use a dependency that you could replace with an afternoon of programming”.

      1. 4

        I would counter that the problem is being wise enough to discern which dependencies really can be reimplemented in an afternoon.

        I would also point out that the “cycle” is not from some hypothetical low-dependency utopia to tons of third-party libraries and back, it’s from NIH syndrome to tons of third-party dependencies and back.

        1. 3

          I would counter that the problem is being wise enough to discern which dependencies really can be reimplemented in an afternoon.

          Thankfully, the cost of answering that question is at most one afternoon.

          1. 3

            From the comments of my post:

            A lot of people are objecting, “What if you estimate wrong, and it takes more than an afternoon?” This objection is bad.

            It is not possible to add a new dependency in less than afternoon because you need to evaluate alternatives, test it, learn how it works, make sure it’s not accidentally GPL, etc. So there are not two methods, the less-than-an-afternoon method and the more-than-an-afternoon method. There are two methods that both take at least one afternoon. If you estimate wrong and you can’t write the code in an afternoon… Then stop working on your handwritten version and find a dependency? But now you know what the dependency is really supposed to do and why it’s non-trivial, so you’re in an even better position to evaluate which ones are good.

            1. 2

              Thankfully, the cost of answering that question is at most one afternoon.

              Or the cost of it is implementing a solution in an afternoon, and then finding oneself the subject of a “Falsehoods programmers believe about…” article because the problem domain turned out to be more complex than an afternoon’s exploration revealed.

              There are no objectively-correct universal stances. Implicitly distrusting/avoiding dependencies is just as wrong as implicitly trusting/embracing them.

              1. 1

                Strongly disagree here. It is very easy for reimplementing something in an afternoon to appear to be successful, only to cause difficult-to-debug problems further down the line.

                Knowing when that is likely to be the case is a function of experience and expertise.

                1. 3

                  I could say the same to you: it is very easy to take on a dependency that appears to solve your problem, only to cause difficult-to-debug problems further down the line.

                  Easier in fact than implementing something in an afternoon and fail to foresee the potential problems: because unlike my own code, I don’t know what’s behind the dependency. So I have to rely on recommendations, reputation, outside appearances. And if it’s not famous I’ll likely need to look up the code itself, and that takes quite a bit of time too.

                  And let’s be honest, if someone implement something in an afternoon, and fail to see the technical debt they’ve just created, they’re not very good. I have little sympathy for tactical tornadoes.

                  1. 1

                    I could say the same to you: it is very easy to take on a dependency that appears to solve your problem, only to cause difficult-to-debug problems further down the line.

                    Sure. I don’t disagree with this at all. I’m not arguing for more dependencies. I don’t have any particular dog in the pro-dependency vs anti-dependency fight.

                    I was only replying to one very specific argument you made which sounds plausible on the surface but is clearly false when given a cursory examination.

        2. 4

          It also sucks when open source maintainers get called things like “uncooperative” when they refuse to add features/fix bugs. Often those with the worst reputation use overly harsh language, but a lot of my favorite open source software is maintained by people who know how to say “no” (the Kitty terminal is one example).

          1. 8

            There are two superficially similar behaviours:

            • Refusing to add feature X.
            • Refusing to make it possible for people to use feature X.

            There are lots of valid reasons for the first but good projects make it easy to maintain out-of-tree extensions that can add features that folks miss. SQLite is probably the best example of this. The author simultaneously refuses a large number of feature requests, while maintaining an extension interface that means that most of those features are easy to keep as plugins that you can add if you want.

            LLVM is somewhat at the opposite extreme, changing internal APIs in ways the repeatedly break out-of-tree extensions. Some of these are important for evolving the codebase, others are due to what I think of as the Google engineering mindset: all important code is in the monorepo, if code is not in the monorepo then, by definition, it is not important and it’s fine to break it.

            1. 3

              LLVM is the middle. They don’t care if they break stuff that goes through the back doors. GCC is the opposite of SQLite, they make it difficult to integrate with on purpose for ideological reasons.

              1. 5

                LLVM is the middle. They don’t care if they break stuff that goes through the back doors

                The problem is, with the exception of libclang, everything is ‘the back door’. We don’t have stable (even the kind of vaguely stable ‘we promise to deprecate for one release before we remove’) APIs that you can use for writing an out-of-tree front end, pass, back end, and so on (the C bindings are almost adequate for writing a front end). Even where we have plug-in interfaces, we make precisely zero guarantees that your code will work in six months.

                This ends up being very bad for the project overall, because there’s a lot of out-of-tree downstream code. Most of this doesn’t bother tracking main, it waits for a release and then gets updated, sometimes skipping releases. As a result, we don’t get any benefit from the huge test coverage that this code would provide and instead get bug reports saying ‘something broke in the last year’. If we had more stable APIs, most of these downstream projects could do nightly CI runs with the latest LLVM and we’d get bug reports immediately after something broke.

          2. 3

            Its interesting to see the tension out in the community. Some of us, like this article, are arguing for more focus, more modularity, more maintainability, more re-use. But there is also a camp that misses the “good old days” where every C framework contains the kitchen sink and you just need one so file as a dependency.

            1. 3

              the “good old days” where every C framework contains the kitchen sink and you just need one so file as a dependency

              A couple of years ago I worked on a project where including the whole of dlib was easier than rewriting/extracting a machine learning algorithm contained in it. As a bonus we got a whole network stack, an GUI library, an XML parser, and a bunch of compression algorithms.

              The project ran fully as a web backend.

              1. 2

                meanwhile I’m sitting here with my 500+ packages for a rust app that is basically a CRUD, including markdown rendering, an ORM builder and some templating

                but I can tell you half of these are duplicates regarding different features or versions of the same crate

                guess it’s many “do one thing”s and most of them “do it right”

                1. 4

                  Counting number of packages is mostly meaningless, because they vary in size and complexity. Big numbers imply bloat and excess, but whether that’s true is not possible to tell based on the number alone.

                  reqwest will pull in 150 packages, but if you look at them they’re all for reasonable things a modern full-featured HTTP client needs: HTTP protocol implementations, handling of sockets, URL parsing, TLS, gzip, brotli, JSON, base64, punycode, mime parsing.

                  OTOH curl is one dependency. It seems super frugal to use one dependency instead of 150. But did you know curl supports email? and gopher, ssh, samba, ldap, and more. It still has hundreds of “packages” worth of code, from many different authors, it’s just all bundled in the way you don’t see.

                  1. 4

                    I agree with your point, but not with your example. curl is objectively huge but it’s also a poster child for projects that are well-maintained, especially regarding backwards compatibility. I think I don’t even remember a single time where a bumped curl dependency broke anything, so (from me) they kinda get a free pass.

                    1. 1

                      but that’s just another illustration of how problems with dependencies can’t be reduced to a single number, and avoidance of dependencies may not make sense.

                      If you tried to replace curl with a “less bloated” library, you could have more issues with it. Or if you went all the way to claim you have “no dependencies!”, you’d have to reinvent a ton of code, likely with a much worse implementation, and a ton of headaches maintaining it all by yourself.

                    2. 1

                      OTOH curl is one dependency. It seems super frugal to use one dependency instead of 150. But did you know curl supports email? and gopher, ssh, samba, ldap, and more. It still has hundreds of “packages” worth of code, from many different authors, it’s just all bundled in the way you don’t see.

                      Numbers obviously don’t paint the whole picture, and definitely not about bloat, but that doesn’t mean none of the small parts of the picture that they do paint aren’t relevant. In curl’s case, if you do work on a project that really does need email, gopher, ssh, samba and LDAP, having one dependency that supports all of them vs. one for each does mean that:

                      • They expose a similar interface where possible, or at least generally use the same idioms, and share a lot of backend code
                      • They generally have the same approach to feature lifetime (and, importantly for long-lived projects, deprecation)
                      • There’s no effort in integrating them (they’re already integrated!)
                      • You only need to monitor a single project’s mailing lists, you only deal with one release stream
                      • If you commit fixes upstream, you have a single set of policies to deal with

                      I don’t miss working with things like Boost, but depending on what’s in it, staying on top of a 100+-package dependency chain can be really exhausting in my experience.

                2. 3

                  In my experience Open Source, especially when independent does a way better job at this than corporate environments or projects backed by companies.

                  Sometimes these things lead to forks though and that can be really detrimental on the ecosystem side, when some things (documentation, extensions, third party software, etc.) supports both, some doesn’t and it gets complex once they diverge until one of them wins, sadly usually the short term benefits one, so the fork.

                  1. 3

                    In my opinion, saying “no” is just as important in commercial software. But generally your audience is much smaller in the commercial setting; and you’re probably much more familiar with them. They’re either a product manager, tech lead, or even a company leader if the business is quite small.

                    What’s important in both commercial and open-source settings is saying “no” well. It really is a skill that requires a lot of refinement. Just like telling kids “no” without a solid explanation, users and co-workers will push back on your “no” if it lacks good reasoning, good taste, or simple civility.

                    Say “no” often and be nice about it.

                    1. 2

                      The most under written project file is the Contributing.md. Most examples never talk about where the bar is for suggestions, what type of responsiveness should happen, and what size of commit to push. This lack of clarity is often expressed by repositories with aging, unloved, unreviewed pull requests.

                      1. 2

                        no is temporary, yes is forever.

                        1. 1

                          I wrote a similar piece of this topic for GitHub’s ReadME project: Knowing when to say ‘no’

                          I took the position that “saying ‘no’” is an attribute (and skill!) common to all software maintenance, but it’s particularly important for open source projects where the ratio between maintainers and contributors is especially high.