My issue is that I believe a software engineer should be able to apply known, repeatable, principles and practices to a software endeavor in order to produce a predictable outcome within determined constraints.
Where does that exist in our industry, other than in non-complex systems?
99% of the time, complex systems come about when you let someone fuck around with new tools instead of solving a problem.
Indeed! Needlessly complex systems come about when you allow that to happen in pretty much any industry. The complexity of a problem tends to translate into complexity of the software that solves it, much like it tends to translate into complexity of the machinery that solves it. E.g. tax software is very complex because tax legislation is very complex, even though it’s “just” arithmetic, much like photolithography equipment is very complex because the sequence of chemical and physical processes it steers is very complex.
Granted, not all problems being solved stem straight from the “real-world” application of a program. Things like allowing only people with the right clearance to view some documents (“security”) are also legitimate problems that a system may need to solve. The bloat line isn’t always easy to draw.
But complexity additional to that of the legitimate problems being solved is entirely self-inflicted, and often arises because nobody in charge or anything has any understanding of what’s important and what isn’t, so they let everyone – or, worse, they insist that everyone – fuck around with new tools.
There’s a glimpse of this in the industry already. It’s a niche but it’s there. Formal methods.
For example, there are relatively big systems (tens of KLOC) being built using Dafny or Spark.
They use techniques that are repeatable, simple and pretty well understood (imperative languages + contracts).
Once there’s an incentive to use them more, I guess bigger systems will be built by composing simpler verified parts. The biggest problem is cost. It’s 3-6x more costly to build software like this, and it’s much slower.
One open problem is that most proofs don’t compose, as I understand it, so you can’t just use small verified parts and compose them into a larger verified whole, you have to re-verify the whole as an entire system, at least for some classes of formal methods. @hwayne I think has more details there.
Absolutely. Dafny for example only carries postconditions forward to keep tractability while composing things, but that looses some information so it’s a compromise. In practice, I think it works quite well.
I think that is an interesting point, but is that glimpse on the fringes? A much more pressing need is surely to find a predictable way to model a complex domain problem in code? When I started programming about 40 years ago the question of the day was how to represent business logic and constraints outside of a data model. Stored procedures, or… what? In some ways nothing has really changed, albeit instead of stored procedures the engineering of the day is transactional scripts organized using functional decomposition which rapidly approach a big ball of mud as complexity increases. Hmm, sounds just like 30 years ago, but without the “agile” and “services” terminology that provide a diversion today.
Formal methods are I think answering a question of how, where the problem is more one of “what”?
“The hardest single part of building a software system is deciding precisely what to build.”, Frederick Brooks – “No Silver Bullet”
I’m going to make a leap of faith and assume you’re saying that because you feel that other “engineering” fields have those properties…they really don’t, in my experience working closely alongside mechanical, electrical, and chemical engineers.
I can’t really claim the “crossover” badge: by the time I got to university I’d known I wanted to be a programmer for like ten years and I was already getting paid to write software. I wanted to get a degree in EE because I hoped a more thorough understanding of how computers are designed and built would help me be a better programmer. I liked it more than I expected and ended up doing more of it than I expected, but I don’t have that much EE experience.
However, from the little I had the chance to practice, and after years of being the bridge between software and hardware teams, I am 100% behind this line:
Nobody I read in these arguments [about how “software engineering” is not “engineering”], not one single person, ever worked as a “real” engineer. At best they had some classical training in the classroom, but we all know that looks nothing like reality. Nobody in this debate had anything more than stereotypes to work with.
All that stuff about “requirements don’t change”, “predictability”, “repeatable patterns and best practices”, “applying scientific knowledge in an exact manner”, to name just a few of the things I’ve heard on HN? Absolute bollocks. I could address all of them in detail but it would be a pretty long post and I’ve done it so many times I would get bored. But I think it’s absolute, total, complete bullshit. The only reason why it’s repeated so often is that, lots of times, people who have no idea how to make software end up running software teams, and they need to explain (both among themselves, and to higher-ups) how come they screwed it up when the eggheads in the other building didn’t. So lots of people, over time, needed to come up with bullshit reasons why “it’s just not like that” in order to cover the fact that, while it is indeed not quite like that, figuring out how to make it work nonetheless is literally their job.
Virtually all of these arguments could have analogies in every other field of industrial activity. There are a million reasons – from environmental concerns and legislation to the time scale of the phenomenons they involve – why chemical engineering is nothing like electrical engineering, or most other branches of engineering, for that matter. But nobody uses that to claim that chemical engineering is somehow not engineering, or if they do, nobody takes them seriously, in any case.
That not all software development activity “counts” as engineering is a whole other story, and also totally true, and it’s confusing for the same reasons as it’s confusing in all other fields. Soldering and building prototypes and whatever is an integral part of (some branches of) electrical engineering activity but I’m probably not doing much engineering when I’m repairing my bass combo, no matter how much soldering is involved.
But either way: don’t buy into this bullshit, don’t devalue your activity, and especially don’t let clueless smartasses devalue your activity. If you see some smartass Agile consultant trying to consultsplain you why software engineering is not really engineering, always remember that they’re probably just as clueless about any branch of engineering as they are about writing software in the first place. Nod, smile, and pretend they’re reading you your zodiac, because that’s exactly how reliable their information is.
I’ve never understood this question. It seems very obvious to me that software engineering is solving engineering problems in the domain of software. You could use some equivalent definitions, but I think the divide between casual software development and professional software engineering is bigger than most people think.
I can teach anyone how to write a TCP server that connects to a PostgreSQL database. They could probably do it in an hour or two. Having them scale that server to 200,000 concurrent users is an entirely different problem.
Much of the math that mechanical engineers use is continuous math. This is where we work over a continuous domain, like real numbers. Things like calculus, trigonometry, and differential equations are in this category. …
In software, we don’t use these things, leading to the conception that we don’t use math. But we actually use discrete math, where we deal exclusively with non-continuous numbers.
No, you don’t use continuous math in your software. Plenty of people do! Do any rendering and you’ll at the least be doing a good chunk of linear algebra.
Software development is all about the unknown. The production process for software is trivially easy—just copy a disk or CD. The software engineering metaphor fails because we understand production, a mechanical task, much better than we understand design, an intellectual task. - Software Craftsmanship
This screams ignorance. Packaging software is far from trivial. I’m sure the book has more context, but the ability to copy/paste our final files doesn’t contradict software engineering being a field of engineering. Have you ever copied an executable from a Windows machine to a macOS machine and had it run fine?
Most people don’t consider a website “engineered”.
If that website is serving a country’s worth of people, you can confidently assume there were engineering problems being solved.
If you did actual engineering work, switched to software, and then disagree that software is engineering, then according to the article you’re in the minority of what the author terms “crossovers” :P
This might be a really tired take, but to me engineering is about constraint solving, and particularly reconciling laws of nature with human requirements while not blowing your budget. Perhaps that only covers one activity that makes up engineering.
By the time this essay was first posted here, I held kind of a neutral stance on the subject; I argued that some software can be considered as “engineered” but, at the same time, there is no entry barrier to those who wish to experiment and produce software with no regards to quality, scope or cost. Now I hold a more cynical view of the matter, which I can only hope will lead to its better understanding in the near future.
So, such barriers do exist in other fields, but they are mostly artificial. If you grok the physics, you can build an airplane yourself—it might be safe to fly it or not. But you’ll need loads of cash, and you’ll not get an FAA certification that easily though. Poor analogies like this could have been concocted with any other engineering area you could think of. Although somewhat dim, these artificial barriers are a crucial part of the modern understanding of what engineering is but, at the same time, they are not part of any conventional definition of engineering. This environment, provided by the synergetic cooperation between oligopolies and red tape, has led other fields to develop their own strategies to help companies and professionals protect themselves from liabilities.
The software industry didn’t need to go such great lengths, as it could get away with it by simply adding a couple of waivers to product licenses. It was a good compromise, even from an engineering perspective, when all that was at risk was a company’s annual balance sheet or an end-of-semester assignment. It will probably continue that way until, say, a landmark case renders those void because a software-controlled piece of critical infrastructure has failed and put lives or properties at risk.
This same article was already posted at a slightly different URL.
My issue is that I believe a software engineer should be able to apply known, repeatable, principles and practices to a software endeavor in order to produce a predictable outcome within determined constraints.
Where does that exist in our industry, other than in non-complex systems?
IME, “non-complex systems” are largely the ones built by people who treat it like engineering.
99% of the time, complex systems come about when you let someone fuck around with new tools instead of solving a problem.
Known, repeatable methods exist for virtually every aspect of programming, at virtually every layer.
Indeed! Needlessly complex systems come about when you allow that to happen in pretty much any industry. The complexity of a problem tends to translate into complexity of the software that solves it, much like it tends to translate into complexity of the machinery that solves it. E.g. tax software is very complex because tax legislation is very complex, even though it’s “just” arithmetic, much like photolithography equipment is very complex because the sequence of chemical and physical processes it steers is very complex.
Granted, not all problems being solved stem straight from the “real-world” application of a program. Things like allowing only people with the right clearance to view some documents (“security”) are also legitimate problems that a system may need to solve. The bloat line isn’t always easy to draw.
But complexity additional to that of the legitimate problems being solved is entirely self-inflicted, and often arises because nobody in charge or anything has any understanding of what’s important and what isn’t, so they let everyone – or, worse, they insist that everyone – fuck around with new tools.
There’s a glimpse of this in the industry already. It’s a niche but it’s there. Formal methods.
For example, there are relatively big systems (tens of KLOC) being built using Dafny or Spark.
They use techniques that are repeatable, simple and pretty well understood (imperative languages + contracts).
Once there’s an incentive to use them more, I guess bigger systems will be built by composing simpler verified parts. The biggest problem is cost. It’s 3-6x more costly to build software like this, and it’s much slower.
One open problem is that most proofs don’t compose, as I understand it, so you can’t just use small verified parts and compose them into a larger verified whole, you have to re-verify the whole as an entire system, at least for some classes of formal methods. @hwayne I think has more details there.
Absolutely. Dafny for example only carries postconditions forward to keep tractability while composing things, but that looses some information so it’s a compromise. In practice, I think it works quite well.
I think that is an interesting point, but is that glimpse on the fringes? A much more pressing need is surely to find a predictable way to model a complex domain problem in code? When I started programming about 40 years ago the question of the day was how to represent business logic and constraints outside of a data model. Stored procedures, or… what? In some ways nothing has really changed, albeit instead of stored procedures the engineering of the day is transactional scripts organized using functional decomposition which rapidly approach a big ball of mud as complexity increases. Hmm, sounds just like 30 years ago, but without the “agile” and “services” terminology that provide a diversion today.
Formal methods are I think answering a question of how, where the problem is more one of “what”?
I’m going to make a leap of faith and assume you’re saying that because you feel that other “engineering” fields have those properties…they really don’t, in my experience working closely alongside mechanical, electrical, and chemical engineers.
No I made no statement about other engineering disciplines.
Release Engineering?
I can’t really claim the “crossover” badge: by the time I got to university I’d known I wanted to be a programmer for like ten years and I was already getting paid to write software. I wanted to get a degree in EE because I hoped a more thorough understanding of how computers are designed and built would help me be a better programmer. I liked it more than I expected and ended up doing more of it than I expected, but I don’t have that much EE experience.
However, from the little I had the chance to practice, and after years of being the bridge between software and hardware teams, I am 100% behind this line:
All that stuff about “requirements don’t change”, “predictability”, “repeatable patterns and best practices”, “applying scientific knowledge in an exact manner”, to name just a few of the things I’ve heard on HN? Absolute bollocks. I could address all of them in detail but it would be a pretty long post and I’ve done it so many times I would get bored. But I think it’s absolute, total, complete bullshit. The only reason why it’s repeated so often is that, lots of times, people who have no idea how to make software end up running software teams, and they need to explain (both among themselves, and to higher-ups) how come they screwed it up when the eggheads in the other building didn’t. So lots of people, over time, needed to come up with bullshit reasons why “it’s just not like that” in order to cover the fact that, while it is indeed not quite like that, figuring out how to make it work nonetheless is literally their job.
Virtually all of these arguments could have analogies in every other field of industrial activity. There are a million reasons – from environmental concerns and legislation to the time scale of the phenomenons they involve – why chemical engineering is nothing like electrical engineering, or most other branches of engineering, for that matter. But nobody uses that to claim that chemical engineering is somehow not engineering, or if they do, nobody takes them seriously, in any case.
That not all software development activity “counts” as engineering is a whole other story, and also totally true, and it’s confusing for the same reasons as it’s confusing in all other fields. Soldering and building prototypes and whatever is an integral part of (some branches of) electrical engineering activity but I’m probably not doing much engineering when I’m repairing my bass combo, no matter how much soldering is involved.
But either way: don’t buy into this bullshit, don’t devalue your activity, and especially don’t let clueless smartasses devalue your activity. If you see some smartass Agile consultant trying to consultsplain you why software engineering is not really engineering, always remember that they’re probably just as clueless about any branch of engineering as they are about writing software in the first place. Nod, smile, and pretend they’re reading you your zodiac, because that’s exactly how reliable their information is.
I’ve never understood this question. It seems very obvious to me that software engineering is solving engineering problems in the domain of software. You could use some equivalent definitions, but I think the divide between casual software development and professional software engineering is bigger than most people think.
I can teach anyone how to write a TCP server that connects to a PostgreSQL database. They could probably do it in an hour or two. Having them scale that server to 200,000 concurrent users is an entirely different problem.
No, you don’t use continuous math in your software. Plenty of people do! Do any rendering and you’ll at the least be doing a good chunk of linear algebra.
This screams ignorance. Packaging software is far from trivial. I’m sure the book has more context, but the ability to copy/paste our final files doesn’t contradict software engineering being a field of engineering. Have you ever copied an executable from a Windows machine to a macOS machine and had it run fine?
If that website is serving a country’s worth of people, you can confidently assume there were engineering problems being solved.
I am and I have the sheepskin (and ring) to prove it.
I don’t work as one on a daily basis, no.
If you did actual engineering work, switched to software, and then disagree that software is engineering, then according to the article you’re in the minority of what the author terms “crossovers” :P
This might be a really tired take, but to me engineering is about constraint solving, and particularly reconciling laws of nature with human requirements while not blowing your budget. Perhaps that only covers one activity that makes up engineering.
By the time this essay was first posted here, I held kind of a neutral stance on the subject; I argued that some software can be considered as “engineered” but, at the same time, there is no entry barrier to those who wish to experiment and produce software with no regards to quality, scope or cost. Now I hold a more cynical view of the matter, which I can only hope will lead to its better understanding in the near future.
So, such barriers do exist in other fields, but they are mostly artificial. If you grok the physics, you can build an airplane yourself—it might be safe to fly it or not. But you’ll need loads of cash, and you’ll not get an FAA certification that easily though. Poor analogies like this could have been concocted with any other engineering area you could think of. Although somewhat dim, these artificial barriers are a crucial part of the modern understanding of what engineering is but, at the same time, they are not part of any conventional definition of engineering. This environment, provided by the synergetic cooperation between oligopolies and red tape, has led other fields to develop their own strategies to help companies and professionals protect themselves from liabilities.
The software industry didn’t need to go such great lengths, as it could get away with it by simply adding a couple of waivers to product licenses. It was a good compromise, even from an engineering perspective, when all that was at risk was a company’s annual balance sheet or an end-of-semester assignment. It will probably continue that way until, say, a landmark case renders those void because a software-controlled piece of critical infrastructure has failed and put lives or properties at risk.