1. 26

  2. 16

    I don’t disagree with the heading…. but I would strongly assert we a VERY BAD at doing “Just” plumbing.

    We have this terrible habit of concreting ourselves into the design suggested by whatever the prebuilt components presented to us.

    We then pat ourselves on the back saying we don’t need to unit test, the prebuilt component is well tested, and we don’t need to think about design, the prebuilt component has thought about that for us.

    And when we decide that component doesn’t fit we rip up everything we have done or insert layers upon layers of kludges to shim in something else.

    And when we want a different UI or backend…. we find we have welded these components into our very architecture.

    At severe risk of enraging DHH further…. I point at this talk as an example of someone who has thought about the issue and has an excellent proposal for resolving (some of it).

    We also forget that retrieving, versioning, licencing, patching, bug fixing and configuring and deploying these pre-built components is hard. This is a large part of what bitbake and OpenEmbedded do for us… and partly what docker should be doing for us.

    Bitbake makes it nice and clear, the “recipe” for grabbing the correct version of the source, verifying it, backing up the original in case the source goes down, verifying the licence, patching, configuring, building, installing, deploying, post installation configuration, ….., for this package AND all it’s dependencies…..

    …that recipe is our source code.

    That is the “crown jewels” of the “just plumbing” era. That is what goes into version control.

    And we do NOT have Good best practices for developing and designing and testing this stuff.

    I really really don’t think this is a solved problem yet.

    Yes, “tinker toy” development as I call it, where we plumb together very large chunks, allowing very rapid development of a huge amount of functionality is the future.

    But we aren’t there yet.

    Not even close.

    Stepping stones and signposts on the journey there are….

    • guix and nixos
    • The reproducible builds project.
    • bitbake and openembedded.
    1. 2

      I think your perspective is correct in a software for developers kind of way. But in production I would say that plumbing and rapid development through external modules is already happening. Any company right now will use npm and many language specific dependency/package managers. In fact, it’s heavily incentized. Your perspective of best practices would typically apply to companies and organizations where time to market is not a significant factor. Which usually means large companies and OSS, but in those cases they would rather do it in-house anyways for other advantages.

      The only portion of software I see this being true is perhaps very specific embedded applications. Such as drivers, industrial controls, FPGAs, etc.

      1. 1

        Oh, I know we all doing it now. Gluing together hundreds of packages and calling it a product.

        I’m just saying we’re all doing it very badly.

        The entire industry is at that painful stage the “handcraft it out of C” part of the industry was at in the 1970’s…

        ie. Groping for what best practice actually looks like, feeling bits of pain here and there, knowing that ten years down the line the pain is going to become excruciating…. but not having and really, umm, solid, principles and tools and practices.

        We’re like the C++ community was prior to STL, and way way prior to the isocpp Core Guidelines.

        We’re at the stage where major players like IBM and Rational defined what they do as best practice and everybody copies that…. only to work out years later that just because you’re big doesn’t mean you’re good at it, just means you can throw hundreds of bodies at it.

        The “IBM” of today is facebook.

        Software in the Large was and is really hard to do Right.

        The era of Software in the Gargantuan that we’re in now will be harder.

    2. 5

      Larger question: why are we proud of this? Do we want programming to just be wiring up components written by 25 year-olds from Facebook with an excess of free time?

      1. 5

        I know for certain I can knock up vast chunks of functionality by gluing together prewritten chunks. I know this, I do this, I know I deliver orders of magnitude more than I could ever write myself.

        I also know we haven’t learnt how to do it well, how to do it in a rock solid, reliable, testable, repeatable way. That video to hexagonal architecture is an example of a way of doing the “glue” part in a rock solid, testable way.

        We haven’t really learnt the best ways. Yet.

        The entire industry is learning on the job. And some of those lessons are going to be really really painful…. Especially for those who don’t recognize that they still need to be learning…

        1. 2

          I know that selfishly I don’t want to wire things up (it’s boring, it’s frustrating etc.), but what is the justification for starting every project by hand rolling your own language and compiler? Surely that wouldn’t benefit anyone but the developer. Even though wiring up components produces software that’s suboptimal in many respects, it’s nevertheless efficient on two very important metrics: cost and development time.

          I’m sure there are exceptions to this (enterprise software comes to mind) but in general, I struggle to make a case against reusing components.

          Looking at it from another angle, we generally want to solve ever more complex problems with software. Managing complexity requires abstraction. Components seem to be the only widely accepted mechanism for creating large scale abstractions. I know this isn’t necessarily the best way, but what is a practical alternative and how can all the existing components be replaced in a way that doesn’t create astronomical costs?

          1. 2

            I’m not arguing for bootstrapping the universe just to make an apple pie. I’m actually a big fan of components. But the view that we’re “just” component wiring plumbers irks me to my core.

            Somebody has to envision the data flow from the user to the app. Someone has to design the interface. Someone has to empathize with the user to discern an optimal workflow. Someone has to also be able to make the machine perform this task in an acceptable amount of time. And someone has to have the skill to design it in a way that doesn’t prevent future improvements.

            I’d argue the utopian vision of software components is already here, where you can drop in various modules and glue them together. Add in an appropriately flexible language, such as JS, and there is very little friction involved overall.

            Also note that the software problem hasn’t been solved, design skills are still needed, and people merely ship faster apps in a more buggy state.

            So, I speak against “just component wiring” in the de-skilling sense if only to say the actual programming part is only a small part of what a programmer does.

            1. 2

              Just playing devils advocate, but how many of us actually have design skills in an engineering sense? To be more specific, how many of us actually design in terms of a definable process? Design is definitely what differentiates a senior and junior but is it something concrete or something more like aesthetics? Another language you learn to talk to other developers.

              It’s interesting because the whole Requirements-Design-Architecture Formalized Design process is not really used anywhere, and places that it is used, design is done by an architecture team and locked away never to be seen by another human.

        2. 5

          Just plumbing… The question I always feel compelled to ask: has the author ever actually done any plumbing? Or at least thoroughly thought about it?

          I have not done much plumbing, but I’ve done my share of home improvement projects and know people with jobs often viewed as ‘boring’, repetitive and non-creative.

          To continue the plumbing metaphor: if you’re renovating, you run into all kinds of non-standard setups, walls that are already tiled, non-standard existing piping, … But even greenfield plumbing is nontrivial and challenging. You don’t get to design the house, they may be pretty late in bringing you in, there will be constraints by wall and floor materials, supporting walls, slightly non-standard requests, etc.

          Every project is different and requires creativity. You’re often better off paying a bit more for a smarter plumber.

          1. 1

            Libraries are the compiler of 2018. Makes things a lot easier, you can always go in and fix it if it doesn’t work, and the developer refrain is that somehow it’s the erosion of the human mind. For historical context this is what developers thought of other developers who used compilers. Yet today very very few people would refer to someone who used a compiler as a simpleton. In fact we’ve learned that having access to a compiler actually allows us to express even more complex abstractions. Yet we still go through the same fearful conversation about libraries.