1. 118

Hello Lobsters!

Back in 2019 I started the “Crossover Project”, where I researched the differences between software and “real” engineering by talking to people who’ve professionally done both. 17 interviews and 12 hours of recorded conversation later, I’m sharing what I found in three essays. The first one is available, while the next two are going up later this week:

(I’m including all the links now because there’s a time limit for editing stories. Those are the links I will put them up at, they’re just not up yet.)


  1. 11

    Fantastic article!

    But please, don’t just update this thread with the new links. I’d rather they also get posted as separate links. It’s easier to catch the new ones if they are posted as their own posts. Plus, the comments will be relevant to each “chapter” instead of just a mishmash of comments in one thread that were posted before we had the full picture.

    1. 3

      Since it’s three related essays coming out in one week, I asked @pushcx whether he preferred three separate threads or one megathread. His preference was for the latter :)

      1. 11

        I’d also like to ask you and @pushcx to reconsider. I don’t want to keep going back to this post (which might drop off the front page after a few days) just to check if the next story is up yet.

    2. 8

      I think this is really good research, it is obvious (if you think a little), that engineering stereotypes are wrong, but that hasn’t stopped plenty of people (myself included) from making bad comparisons.

      I think the section on licensure is lacking some historical context, though. It portrays licensing laws as essentially benevolent regulations to increase public safety or protect public investments, but that ignores the rich scholarship on how licensing and similar regulations have been used to discriminate against and exclude minorities who might otherwise enter an industry.

      We can see this consistently over time with e.g. Jewish folk being banned from many trades in Europe and many Southern USA states using licenses to support segregation. There is competing scholarship for the more recent US case, and I am not an expert, but this rebuttal paper covers some of the common issues and links to lots of seminal work (Was Occupational Licensing Good for Minorities? A Critique of Marc Law and Mindy Marks).

      (Edit: it’s further off-topic, but I should note that the modern story is more complicated and now licensing may help women and black people in the US, though often for racist reasons. (Research Spotlight: Occupational Licensing Reduces Racial and Gender Wage Gaps))

      1. 1

        I think the section on licensure is lacking some historical context, though. It portrays licensing laws as essentially benevolent regulations to increase public safety or protect public investments, but that ignores the rich scholarship on how licensing and similar regulations have been used to discriminate against and exclude minorities who might otherwise enter an industry.

        Whether or not licensing laws are good or bad is exactly the question that section is arguing is irrelevant to the topical question of whether software developers should count as a type of engineer:

        In conclusion: licenses exist because we are part of society and have legal requirements, not because they are essential to what it means to do engineering. So while you might want to make software more licensed, as it stands the licensing question doesn’t change the essence of our work.

        I wouldn’t say it’s lacking historical context, but rather that historical context about how licensing laws have been used in the past is only relevant to determining whether or not licensing laws are good, which is a question this article is deliberately not dealing with.

        1. 1

          The article says:

          Societies adopt licensure for reasons that are as much political as technical. To see this, let’s discuss some of the history of licensure in the United States.


          While licensure originated as part of cost overruns, it expanded so rapidly for an entirely different reason. Regulations are written in blood. Fields become regulated when the lack of regulation kills people. Until that happens on a wide scale with arbitrary programs, it’s unlikely that we’ll ever see the same licensure requirements for software.

          (emphasis mine)

          In retrospect I was interpreting these statements more widely than Hillel probably intended. Fields become regulated for all kinds of reasons, and also because of deaths. And Hillel does describe some of the history of licensure in the US, just a very small part.

      2. 5

        @hwayne: Is there a way I could pay you some amount of money for these essays? Or would you be willing to put together some swag or the like?

        1. 3

          Thanks for the kind offer! The easiest way to throw money my way would be to buy a copy of my newsletter archives. As a bonus, you get 300 pages of software essays!

          1. 1

            I would buy this if I could put it on my physical shelf without figuring out how to run my printer. I don’t know if paper publishing is easy to do, once you have a virtual book.

            1. 1

              Yes; “print on demand” is the phrase. Not endorsing either, but both Lulu and Amazon offer it.

        2. 4

          Essay number two is now up.

          1. 1

            I’ve only just gone back and read Essay Three.

            I really enjoyed this. Thank you.

          2. 3

            That is something I’ve thought about a lot. The article is therefore a very interesting read. I’m a mechanical engineer myself, who is also very involved into software (both privately and professionally). I would describe engineering as follows: “To design something to an optimum under constraints”. I know this is very broad, but it goes very well along with “everyone can build a bridge, but only engineers can build a bridge which barely stands” to quote Practical Engineering. I think software engineering fits very well with this: Everyone can write software that works, but only software engineers can write software that barely works (clean and minimal code and no excess functionality). On a broader picture engineers among many other things also ensure that this something works for its defined lifespan.

            I’ve one remark about the math part though: Engineers almost solely use discrete math also, because many problems are not solvable otherwise, but you only learn this during the later semesters and many engineers don’t need higher math when working. Continuous vs discrete: A falling ball can be very easily described with continuous math, but if you want to include air resistance you have the go discrete. Hint: Engineers in the field mostly want that.

            1. 3

              I think that this has the potential to really open a lot of doors that software engineers have closed for themselves (or never realized were open) through our perceptions of exceptionalism. By getting a clearer orientation about how our work compares to that of others, I believe this project will facilitate conversations across professions that we would not have realized were even possible before. Congratulations for publishing this!

              1. 3

                Hurray! I’m very happy that you’re publishing these. The history and general practices of science and engineering are fields that interest me deeply, and I think this work on making the comparison between software and other engineering fields is really fascinating. The occasional teases on your Twitter account have been fun to follow.

                Anecdotally, even as a student I noticed that the people who were least familiar with each others’ fields exaggerated the differences the most. For example, I did some grad work at a big state school where people mostly socialized in their own departments, and you sometimes heard people talk about other departments very dismissively, or with an air of mystery. (“What do you think those people do all day, anyway?”)

                By contrast, I got my undergrad degree at a smaller, physically isolated engineering university, where we all more or less had to get to know each other due to sheer proximity. ;-) In that setting, the similarities tended to stand out more than the differences, and there seemed to be little question that software folks “did engineering”.

                1. 1

                  Hm nice that you actually went and interviewed people! FWIW I was not surprised by their opinions. I would definitely consider programming a kind of “engineering”.

                  One way to see this is that you can do it badly! If it weren’t engineering, then maybe there’s no way to do it badly. But we all have lots of opinions on when a program is poorly written. For example, you can call a website that is constantly crashing or slow “badly engineered”; you need apply engineering to rectify that situation :)

                  I think software has the potential to be (and is in some cases) a lot more rigorous than other fields of engineering due to increased use of testing (and mathematical techniques, to a lesser degree). This was a big change even since I started my career, i.e. we didn’t write tests and now we do.

                  I always liked this talk about deterministic simulation and “adversarial” testing of FoundationDB. I think more software projects will start to look like this in the future, and it feels above the level of rigor that you can achieve in the “real world”, mostly due to speed.


                  1. 1

                    Thanks @hwayne for a great set of articles, which I can a cross due to the repost or the first article and hightlights the importance of learning from the pitfalls in other disciplines.

                    1. 1

                      Thank you for doing this research and writing it up! I am looking forward to reading. There may be other terms, but did you happen to evaluate the term “architect?” I feel like it is in a similar boat of being analogous to “engineer.”

                      My personal view is that words do have meaning, but their meaning differs based on context. Likewise, words may be useful to different audiences of people. What is generally always true is needing to explain what I do beyond just the term “engineer” or “developer” or “architect” or “programmer.”

                      This may be a odd analogy, but I think about the term “cook” vs “chef.” Many of the best chefs refer to them as cooks.. so maybe words also attempt to reflect status or humbleness.

                      1. 1

                        “Software architect” is already a title and it’s gotten a bit of a negative connotation.