1. 57
    1. 20

      If a civil or structural engineer were told by their boss to do something so clearly deceitful and potentially harmful to the public on behalf of a big client, they could take their concerns to their professional society, who would protect them and perhaps even take other legal action to protect both the public and the reputation of the profession. In fact, if they didn’t, and one of their colleagues found out, they could be in trouble with their association – which could have real consequences for their career. But if a “software engineer” has such qualms, the best they can do is quit (leaving a vacancy for someone who may not have the same scruples) and maybe, as a stretch, put themselves at risk by trying to raise awareness as an unprotected individual whistleblower.

      That’s why I say we’re not real engineers. It’s nothing to do with what kind of work we do or how we do it. It has everything to do with our lack of professional standing in society at large, and institutions which can defend it. Until we can govern ourselves the way that architects or attorneys or doctors or all the real engineering disciplines do, we are just unusually well-paid labor, externalizing risk onto the public with no effective responsibility, to our collective shame.

      1. 4

        That’s why I say we’re not real engineers.

        I agree entirely - but have largely given up on a battle that most of my colleagues and employers incorrectly consider to be purely semantic.

        I’m also very, very wary of calls for compulsory regulation and registration with professional societies. A lot of what programmers do isn’t engineering, by design.

        1. 1

          I’m not interested in the semantics battle, myself. I’d say it’s still very early days for the professionalization of software, and I’d like to be clear that I’m not calling for anything in particular at present. Nor am I holding my breath. But I do anticipate some change over the span of my own career, and there’s plenty of historical precedent for anyone who cares to look. I spell out my position in some detail in my reply to LibertarianLlama.

      2. 4

        If a civil engineer does something bad, a building collapses and 100 people die.

        A software not-engineer does not have this problem 95% of the time.

        For the OP’s article, the story is a person who was asked to code a marketing website for a drug. Another person suicided while on said drug. Therefore, the marketing website is unethical.

        This is wrong. It could have been that the drug will save 10,000 people for every suicide it causes. It is the responsibility of the patient and their doctors to weigh the risk/reward of a particular drug. Doctors and researchers have a system they build the weigh the risk/reward of drugs and to control corporations that develop those drugs.

        To simply conclude that the drug is unethical, therefore making a website marketing it is unethical, based on one case of a negative outcome, with no mention of positive cases, is simple not a good way to arrive at the conclusion of whether this is ethical.

        That’s why I say we’re not real engineers. … It has everything to do with our lack of professional standing in society at large, and institutions which can defend it. … we are just unusually well-paid labor, externalizing risk onto the public with no effective responsibility, to our collective shame.

        Lawyers are considered bane of civilisations by many people. They build an overly legalised system to extract rent from societies via legal obligations. Why do you consider them to have ‘standing’ in society?

        Is your argument that if programmers get together, create “The Grand Association of Serious and Ethical Programeers”, that we will have standing? Or that we will be ‘serious business’ in the eyes of the public? Or that we will be real white collars? Or a real profession? And that we will be able to defend the honour and dignity of our sir programeers? Will we demand a uniform of t-shirts and hoodies like lawyers do? Because we are serious business.

        Programmers have deliver HUGE amount of value. There’s absolutely no gatekeeping. A random kid with internet access from the poorest part of the world can get a remote job, with salary that will completely transform his extended family’s lives. This is not possible as a lawyer, or a doctor, or an ‘engineer’. No matter how good you could be, without the correct due paid to the correct universities, and going through their gatekeeping processes. It is the realisation of the free market dream, in a way that is potentially completely transformative. But we shouldn’t have that, because some people decided to code a marketing website or a drug that has otherwise been okayed by those doctors and lawyers that you hold in such great esteem, and he didn’t like it that a person who was on that drug died from its side-effect.

        If you only read the news about people drowning, soon you will want to ban water.

        1. 4

          Regardless of whether the drug itself is good or bad, the marketing was deceptive, which both the OP and I see as unethical. Anyone with even a passing acquaintance with the US (and international) pharmaceutical industry knows that they have a long history of doing some incredibly shady stuff, despite being highly regulated. Certainly they love loopholes. But that’s pretty tangential to my point, which was simply about what actions you can take as an engineer given a task that you judge to be unethical, what are the expected outcomes of those actions, and what that says about our societal status collectively, as a nascent profession.

          I get that you don’t like lawyers. Maybe (like some people would say about guns) it would be a better world if they had never been invented, but that’s strictly fantasy. In the real world, I think it’s pretty uncontroversial to point out that they wield significant power, have effective self-regulation and autonomy, and consequentially command respect. Certainly, as a profession, they are nearly as old as medicine, and equally established. It doesn’t matter how you feel about them as a class of people.

          I don’t see any need for your GASEP: in fact, we already have the ACM and the IEEE. And no, I don’t look forward to a world where (say) ACM membership is compulsory for junior web devs. There are several established ways that society regulates dangerous work. Building contractors, for example, must be licensed, bonded, and insured – but that doesn’t replace building permits and inspections. Architects have their own responsibilities. It’s a complex and nuanced topic that might not be the best basis for analogy anyway. What I’m saying is, as society gradually comes to rely more on our work, and the potential for real harm increases, that work will be regulated, one way or another. We can either self-regulate, or we can be entirely governed by legislation and the internal regulation processes of other fields. In practice, it will probably be both, and will emerge gradually over a generation or two, because our field is still too immature to have any real basis for standardizing practices. But I’m from a blue-collar background myself, and I’m in favor of software practitioners having at least some autonomy and professional standing, which requires self-regulation. We’re not all like handymen or union carpenters. At least some of us are doing work that’s more like that done by architects and structural engineers.

          In summary, you argue that:

          1. Most of what we do can’t cause real harm
          2. Doctors and researchers and laws already sufficiently regulate the drug market such that no professional judgement by website developers is needed
          3. The economic benefits of unrestricted access to high-paying tech jobs outweigh any prospective dangers

          I’d say that (1) is becoming less and less true as software chews through the world and digests what it can. As for (2), I suppose we can just wait for web-based pharmaceutical ads to be as regulated as on TV etc, but it’s entirely beside my point. As for (3), I think it’s worth distinguishing between the costs and benefits borne by practitioners and those borne by their employers, their customers, and society at large. A balance must be struck, and that balance may well shift over time.

        2. 3

          If a civil engineer does something bad, a building collapses and 100 people die.

          A software not-engineer does not have this problem 95% of the time.

          I’m pretty sure most of the civil engineer fuckups just result in expensive shore-ups and re-builds. But… I can’t actually back up that claim.

          It could have been that the drug will save 10,000 people for every suicide it causes.

          As mentioned elsethread, this was probably anti-acne drug Accutane.

        3. 1

          A software not-engineer does not have this problem 95% of the time.

          That is why even a senior software is engineer is equal / equivalent to a skilled builder / construction worker. They do not go to jail for the collapse. The Civil Engineer does (depending countries and jurisdictions on the size of what was built, etc).

          So, who’s ready to go to jail?

    2. 9

      There’s a lot of dangerous shortcuts in thought in that article. It’s concerning.

      The implication “the code I wrote killed these two girls and who knows who many others” is strong. But as a reminder, the specific drug was popular with that segment of the population to begin with. There’s also a baseline of people who kill themselves, drug or not. And we know next to nothing about the drug to begin with. It could very well be helping a lot of girls.

      Rather than try and promote some silly Code of Ethics (to delegate decisions and to push guilt into a third party e.g. “it’s alright because the Code of Ethics allows this”), I’d suggest the author to seek help controlling negative thoughts concerning toxic guilt. Understand what the mistake was and what could have done better, and move on.

      From my perspective, the lesson at the core of the article, besides the standard “do not get hung up on mistakes”, has little to do with ethics, but is quite valuable nevertheless: Never do what you actually do not want to do.

      Regret usually is about things that you could have done but did not. It seems contradictory this time around, except it’s a matter of perspective: The author could have rejected the task, and yet didn’t.

    3. 4

      As a matter of medical ethics, I’m not convinced that creating and marketing this drug in particular, or in general any drug that has depression and suicidal tendencies as side effects, is necessarily the wrong thing to do. Lots of useful drugs have serious side effects, and while it’s important for both doctors and patients to be aware of those side effects when making decisions about whether to use the drug, I don’t think the existence of those side effects implies that no one should use the drug. That’s a complicated medical question that depends on how likely those side effects are, exactly how bad they are, and what aliments the drug purports to treat, how effectively it does so, and how bad the untreated effects of that ailment are (additionally I wouldn’t assume that because one person I heard about in the news committing suicide while taking this drug, that implied that the drug was specifically responsible for that suicide, or that even if the drug was responsible for the suicide, that suicide necessarily outweighed the aggregate benefits of using the drug to treat an ailment). In any case, it’s certainly not a question that programmers, as opposed to doctors and medical regulators, have any special insight about.

      Part of the reason the societies we live in have things like medical ethics laws, governmental regulatory organizations like the FDA, drug regulations, and so on, is because the ethical questions about when it is and is not okay to market and sell a drug are complicated and require medical domain-specific knowledge as well a shared conception of the common good to answer. It’s a very reasonable position to follow the letter of the law unless and until some knowledgeable medical authority, or the evidence of my own or my community’s experience with the drug, convinced me that the regulatory system around this specific drug ought to be changed. And I feel like, say, a doctor writing a blog, might convince me that more people would die without this drug than with it, just as easily as they might convince me that the suicidal thoughts side-effect was too serious and the medical regulatory establishment erred in letting this drug be sold at all.

      Given that I’m not convinced the drug company actually acted unethically, or that the laws that permitted them to sell and market this drug should be changed, then why should I expect a programmer to refuse to write code on behalf of such a drug company?

      1. 34

        not convinced the drug company actually acted unethically,

        Presenting a neutral-appearing “find the best drug for you!” informational website and then always giving the same answer with a fake quiz doesn’t sound like a bald-faced lie to you?

        The issue here isn’t that companies are marketing their drugs; it’s that they’re being deceitful lying twats about it. You seem to have completely missed the point. As you mentioned drugs and side-effects is complicated and hard, which is why you shouldn’t create fake “informational” websites with fake quizzes to market your drugs.

        1. 13

          Agree completely with your point, I think it’s quite clearly unethical behaviour.

          The issue here isn’t that companies are marketing their drugs

          It could be one of them. It strikes me as an incredibly odd thing when I see American TV and am not only bombarded with drug ads, they are targeted at the patients rather than medical professionals.

          1. 1

            I’ve been thinking about this too. It’s also super weird to me that there’s the whole list of side effects at the end… shouldn’t my doctor be telling me that and not the ad? What is I didn’t see the ad? I’m sure there’s some baroque liability reason they have to do it, but it’s dumb. (American here, FWIW.)

      2. 5

        I haven’t worked on code that’s specifically related to drugs and medication, but I have worked on medical devices so I guess my opinion is… partly informed? :). I don’t have experience bringing drugs to market but I’m somewhat familiar with the regulatory system involved.

        There is no question as to the fact that drugs that have certain side effects should be allowed. Practically all of them have side-effects. Even “natural” medicine, like various plants and whatever, have side-effects and can interact with other drugs, “natural” or not. That’s why the side effects are listed on the fine print – so that physicians and patients can make an informed decision about these things, and so that reactions can be properly supervised. How efficient the fine prints are is another debate, but I think we can safely argue that the benefit of some substances can outweigh the risk of side-effects, as long as administration is properly supervised and so on. For example, if a drug can cause depression and anxiety, a doctor can recommend close monitoring of a patient’s mental state, by another doctor or even a psychiatrist if necessary, especially if they lack a support system (if they live alone, secluded etc.) or have a history of depression. Or they may avoid that drug altogether if possible.

        However, uninformed self-medication is also a thing. That’s part of the reason why some drugs are only issued based on prescriptions, and why you’re supposed to keep some of them out of children’s reach and so on. It’s a very real problem, especially when it’s related to drugs for affections that carry some form of social stigma (mental illnesses, STDs), or for particularly difficult age ranges, where people have difficulties seeking help. Depressed teenagers, for example, are not very likely to go to adults for help, especially if their depression is fueled by adults in their life, like abusive parents.

        “Proving that you have a prescription” over the Internet was pretty easy to do twenty years ago (and I think it still is in some cases). You can often use someone else’s. A teenage girl can generally use her mother’s prescriptions pretty easily, for example.

        Now, of course, there’s only so much you can do to prevent self-medication by uninformed people. At the end of the day, if people think it’s a good idea, they’ll get their stuff one way or another. You can’t keep all drugs under lock and key in a safe and not hand them out unless someone brings in their doctor and three independent witnesses to confirm that they need the drug and that prescription they have is real. You print out (or the FDA makes you print out) big warnings, the drug can only be sold to prescription holders under specific conditions etc.. There a point past which you can’t do much to prevent self-medication.

        But acting in a manner that encourages self-medication – deliberately eschewing a physician’s ability to supervise medication and the patient’s evolution – is absolutely unethical. It’s akin to going into a hospital, leaving a bunch of pills on the table, and telling people to help themselves if they want, as long as they don’t tell their doctors about it. Doing so against a vulnerable age group makes it worse, too. Especially if the targeting deliberately exploits a prevalent vulnerability (edit: someone here mentioned Accutane – that was my guess, too, but I was hesitant to call it out, since the original author didn’t, and I don’t know that much about what was allowed in Canada twenty years ago. Accutane was a drug that was meant to help with acne. Yep.)

      3. 4

        Based on timing, target audience, and the issues it caused, it sounds an awful lot like Accutane. That’s a since-discontinued drug for treating acne.

        That aside, I think I would expect myself to refuse to write the code in question because:

        Remember, this website was posing as a general information site. It was not clearly an advertisement for any particular drug.

        and

        “Well, it seems that no matter what I do, the quiz recommends the client’s drug as the best possible treatment. The only exception is if I say I’m allergic. Or if I say I am already taking it.”

        and

        “Yes. That’s what the requirements say to do. Everything leads to the client’s drug.”

        Quite apart from any ethical judgement about what suicide frequency is acceptable as a side-effect for an acne drug, I’d expect myself to refuse to write code whose purpose is to deceive the public into making a particular health decision for the benefit of my client.

        1. 5

          I’d expect myself to refuse to write code whose purpose is to deceive the public into making a particular health decision for the benefit of my client.

          I would too. But I also remember how I was a lot more naive when I started coding, and I would not be surprised if I wouldn’t have picked up on this, just like the author. You expect that, if this was not okay, someone else would have stepped in already. It comes as a shock when you find out that “someone else” should have been you.

          1. 3

            Yes. I should have said “I would now expect myself…”

            I can’t claim with any certainty that I would have caught on 20 years ago.

        2. 2

          Somewhat off-topic, but Accutane isn’t discontinued, although according to Wikipedia the original manufacturer no longer produces it - was that what you meant?

          It is highly controlled in (at least) the US though. You have to get monthly blood tests to make sure it isn’t killing your liver, and if you can get pregnant you have to be on (IIRC) at least two forms of birth control. The latter is why it’s so controlled - it causes really severe birth defects.

          1. 1

            I didn’t realize anyone had picked up the manufacturing. Wow. I remembered it as having gone away.

            1. 1

              Yeah. I took it in 2015, which is how I know. I switched away from Accutane halfway through though because it was cheaper to go with a generic version, which was marketed as something else but had the same underlying active ingredient (isotretinoin) - maybe that’s what you’re thinking of? When I was reading about it last night Wikipedia said the original manufacturer shut down production because cheaper generic versions had become available (and because of lawsuit settlements over side effects…), so it’s unclear to me as to whether in 2020 it’s still actually available under the brand name “Accutane”.

    4. 2

      I can sympathize with the author.

      Long ago, in a medical setting, I participated as an analyst in a study that charged patients a “retainer fee” who would have ordinarily fallen under the hospital’s charity policy. Results revealed that a significant number of people were subject to this fee while still paying full price, out of pocket, due to being at 400%+ of the Federal Poverty Level. This work is published, and I – as a data analyst – saved the paper from a retraction by arguing with a department chair for an hour about external validity. It didn’t change the results, but entire providers had been omitted from the EHR pull, which was corrected pre-publication.

      I am still angry and upset about my participation to this day. People in need – predominantly suffering from depression, anxiety disorders, and metabolic disorders, not to mention pregnancies – were told to pay more than $100 extra at the point of care in order to be seen, or were diverted. This was deeply unethical and I regret my participation.

      I’ve considered writing to the journal to try to lessen my own feelings of guilt, and may well do so. I don’t know if it hurt anyone. I don’t know if a depressed patient walked out after that and committed suicide. Nobody else does either.

    5. 2

      I can honestly say I’ve never written code like this, but then I recognize that I’m very lucky.

      The one time I was ever asked to do anything overtly deceitful in my career, it was my first job out, part of which was doing tech sport for a company that sold TCP/IP co-processors.

      The guy who was the tech brains of the company (only person who knew the embedded OS the co-processor ran in) asked me to lie to a customer repeatedly around their requests that we fix a super show stopper bug.

      I capitulated a couple of times, and then, when I came to the boss a third time asking about the bug, he said “I’m not gonna fix it” - so I explained to the client in full detail everything that happened including what he’d said.

      I expected to be fired on the spot, but it actually earned the respect of the boss and the multi-million dollar client.

    6. -3

      It’s easy to make an argument that I had no part in it at all.

      No it f****** isn’t.

      1. 19

        I think you misread that section. He wasn’t trying to excuse himself. The full text of that paragraph was:

        It’s easy to make an argument that I had no part in it at all. Still, I’ve never felt okay about writing that code.

        He was saying that people will try to rationalize that sort of behaviour, because it seems rationalizable. But it’s not.

        1. 7

          Thanks for nudging me to think this through a bit more and make my point more clearly.

          I understand that the point is the most people will rationalize this, and I think many people have to do this day in, day out in order to not go insane. I want to make the point that to describe this as ‘easy’ is a concession, a misdirection and a simplification. It’s actually not easy (e.g. refer to the mental anguish in the article, ‘shame’ is a powerful thing here), but describing why it is not easy but yet comes to be accepted is the actual issue the article hints at but does not go anywhere near addressing: specifically what is it about our current social context that allows, facilitates and even promotes this ‘rationalization’.

          I think to make that statement more accurate it needs to be along the lines of:

          ‘It’s a socially accepted norm for me to say I had no part in it at all and for most people to agree with that and excuse me from any responsiblity at all’.

          I want to make the point more clearly that just because we all do it, all day long and it’s accepted and a social norm, does not mean it’s easy. This is an internal conflict where you understand that you’re doing something wrong but choose to do it anyway for a socially imposed or practical reason that you feel unable to get out of, for example you feel you must do this in order to advance your career or social status, or you need the money from this work to feed your family, etc.

          This conflict has a significant mental cost which we bear individually and collectively in order to maintain a horrifically broken society which requires this to appear to be accepted and ‘easy’ in order to maintain its current state.

          1. 1

            I don’t think your rewrite is any more accurate. It’s not only socially acceptable, but there is also a strong pull in all of us to justify our actions when they might be perceived as wrong when viewed through a particular lens. In fact the “easy” thing is often to rationalize those away rather than deal with them. I don’t think we need to blame society for this. Society may be a potential solution by setting cultural standards but the cause is inside the individual.