1. 41

  2. 21

    I hope to see more of this — if workers with as much leverage as we have don’t speak up against technology we create being used for evil, we can’t call ourselves engineers.

    1. 13

      Relying on morality when incentives go the other way does not scale.

      1. 6

        Exactly. It has to be a large number of people that damage their mission directly or indirectly with media pressure. Otherwise, it’s something with no impact. At least people are following their principles, though.

        1. 6

          It has to be a large number of people that damage their mission directly or indirectly with media pressure.

          Can you trust an engineering company who ignores the opinions of its engineers?

          We are talking about one of the most celebrated company of western economy, often cited as an example of excellence.

          Leaving Google for ethical concerns poses a serious burden on the employment of these engineers that will probably be marked as dangerous employees for the time being.

          We can assume that this is something they knew, as Google don’t hire dumb guys.

          So why they quit?

          My bet is that the militar use of the Google’s artificial intelligence technology is so dangerous that these engineers felt obliged to leave the organization beyond any doubt.

          Otherwise, it’s something with no impact.

          Well, it’s a first step.

          And a courageous one.

          Its impact goes beyond the worldwide image of Google, beyond the direct issues in their production line.

          It is an example.

          1. 4

            Can you trust an engineering company who ignores the opinions of its engineers?

            It doesn’t matter. What matters here is (a) the companies’ goals/incentives, (b) how successful they are at achieving them, and (c) if a tiny number of engineers quitting changes that. Note that (b) includes implicit support by the many people who use their products and services voting with their wallet. The stuff in (a) means they’re anywhere from apathetic to damaging for a lot of ethical issues around privacy and making money. Due to (b), actions to damage them have to put a huge dent in that or make them think it will. (c) doesn’t do that. So, (c) is probably irrelevant to Google. The article itself says as much:

            “However, the mounting pressure from employees seems to have done little to sway Google’s decision—the company has defended its work on Maven and is thought to be one of the lead contenders for another major Pentagon cloud computing contract, the Joint Enterprise Defense Infrastructure, better known as JEDI, that is currently up for bids.”

            I gave them credit in my other comment for standing up on their principles. That’s respectable. It’s just that a “dozen” or so people quitting a company with over 70,000 employees with people waiting to fill their positions doesn’t usually change anything. They’d instead have to campaign in media or government aimed at stopping those contracts or drone operations. At least half the voting public and current President support military action overseas. The other half didn’t convince their prior President to stop drone use or strikes. There are also not large swaths of Google customers threatening to stop using Google Search, Gmail, etc if Google doesn’t turn down government contracts.

            So, quitting over this is pointless if the goal is to achieve something. At best, it’s a personal decision by those individuals to not be involved in something they disagree with that’s going to happen anyway. That’s fine but practically a separate thing from ending these contracts. If anything, we’ll just get a shift in Google employees from those who might leave over the contracts to people who range from favoring them or just griping about them continuing to work there. I think most will be in latter category.

            1. 2

              It’s just that a “dozen” or so people quitting a company with over 70,000 employees with people waiting to fill their positions doesn’t usually change anything.

              The fact is that fewer talented people will want to fill their position.
              This is a pretty serious issue, if engineers are the core resource of your company.

              Now, I’d guess most Google engineers don’t feel as important to the company as they feel the company is important to them. This happens in many companies, and I would guess Google has turned this kind of internal narrative into an art.

              The fact is that, instead, Google literally would not exists without those engineers.

              These few have shown exactly that: that working in Google is not that important.
              It’s a matter of time, but if Google do not take serious actions to avoid this general wake up, other engineers will follow. And the same might happen in Facebook, in Apple and in many other smaller IT companies.

              On the other hand, in Europe and everywhere else, people will start to ask why engineers from a company that operate in their territories, are so afraid for what the company is doing, to quit. To avoid the risk of being associated with the company future. To avoid sharing its responsibility.
              Politicians will be less friendly to a company that might be doing something really evil for a foreign state.

              I agree that more engineers should follow their example, but I know that life is not that easy.
              However people continuing to work there might organize to keep the company “on track”, and this might lead to the creation of a labor union.

              1. 4

                The fact is that fewer talented people will want to fill their position.

                You have to prove that assumption. Google changed their Don’t Be Evil motto doing sneakier and sneakier stuff overtime. They’re a surveillance company that hires brilliant people to do interesting work for high pay and good perks. They’ve had no trouble that I’ve seen keeping new people coming in. Status quo has the evidence going against your claim: it’s a shady, rich company with in-demand jobs whose shady activities haven’t changed that for years. There’s also nearly 70,000 workers mostly in favor of it with more trying to get in.

                “However people continuing to work there might organize to keep the company “on track”, and this might lead to the creation of a labor union.”

                That’s a different issue entirely. Given I am in a union, I think it would be cool to see it happen. Unlike OP topic, that could happen with higher probability. Silicon Valley will do everything they can to stamp it out in mean time, though. Still a long shot.

                1. 0

                  The fact is that fewer talented people will want to fill their position.

                  You have to prove that assumption.

                  Not an assumption, but a deduction: people avoid cognitive dissonance, if possible.

                  A dozen people leaving a company cause of ethics, means that such company forced them too high on cognitive dissonance, and this will make Google relatively less attractive, in comparison to the alternatives: a talented engineer want to fix problems, not fool herself to avoid the pain of contradictions.

                  Our brain consume around 20% of our energy, after all.

                  This is the same reason that make me guess others will quit Google in the future.
                  Because now they have a new thinkable precedent.
                  A new, effective solution to reduce their cognitive dissonance.

        2. 2

          I agree. But we also can’t rely on companies that we don’t own to incentivize us to act in a moral fashion – engineers need a governing body for that.

          1. 1

            What about entering both US political parties and changing the policy? If you believe that killing people is wrong, maybe make it a law?

            Sometimes the only way to advance your field is to step out of it and fix the external systems. And war zones are definitely not a good environment in which to build global information network to advance everyone’s wellbeing…

          2. 1

            I think it’s definitely a factor. Many prominent business people would not like to be associated with payday loan companies, for example.

            I think this is less about being the silver bullet for problems, and more about being one of the 20 or 30 things we need to be doing to make the world A Better Place(TM)

          3. 13

            We can’t even speak up for honest pay for an honest day’s work–and that’s a lot less subjective than some arbitrary definition of “evil”.

            1. 4

              Why not both?

              1. 4

                At least the “evil” one is super cloudy.

                Say you are an engineer working at a company that builds control software for missiles. You are a pacifist, and so you decide to introduce a minor bug (or fail to patch a discovered bug) that causes the missile to not detonate when it lands.

                • Are you good for not facilitating the loss of life?
                • Are you evil for misleading your employer about the labor of yours that they’ve purchased?
                • If the missile lands on a poor grunt and severs their legs causing them to bleed out over minutes instead of detonating properly and just kinda instantly killing them, are you evil for prolonging suffering?

                That’s just scratching the surface of morality in engineering.

                1. 6

                  That’s fair – and I should’ve been explicit earlier. I believe that there are (at least) two moral guidelines that should be taken into account.

                  The first is a professional code of ethics, similar to what ACM has here. Of course even this is cloudy – for example, in my opinion 1.2 “Avoid harm to others” would necessarily preclude working for a missile manufacturer in the first place. At the very least, if one views missiles and missile software as being a necessary “evil”, safeguards should be put in to protect human life at all cost, etc. etc. The minutiae of the professional code of ethics can and should be rigorously debated, because it provides a minimally viable base for how we should conduct ourselves. So for example, the question of whether or not working in the weapons manufacturing industry truly violates rule 1.2 should be an explicit discussion that is had in a professional organization (not a workplace per se).

                  The second guideline is in line with your own personal moral code. This is important because it provides for people who are religious (or not) or any other number of cultural influences that have caused a person to believe what they believe today. This, of course, has to be superseded by the professional code – for example, if I personally believe that discrimination based on what TV shows you enjoy is okay, that doesn’t mean that my personal morality should define what happens in a professional setting. But in the hypothetical case you provided, even if I don’t feel that writing that software goes against a professional code of ethics, if I am a pacifist, it goes against my personal code. I know from the professional code that purposefully writing bad or buggy software is wrong, and so my only option is to find another job in which both my personal and professional codes of ethics can be upheld.

                  1. 9

                    Why discuss an unlikely hypothetical rather than the issue at hand? Why the need to logically define evil beyond any confusion? This is not even possible in the general case for anything. Can you logically define ‘fun’ such that everyone agrees? At the end of day, evil means what people talking about it think it means, and it’s better to work off of that than to halt all discussion until we achieve the impossible task of absolutely grounding natural language in logic.

                    1. 6

                      It’s precisely because evil is so ill-defined that talking about it is difficult. As @mordae points out, it’s more effective to talk about other incentives.

                      And again, I’m not saying “halt all discussion”–quite the opposite! I’m saying that the issue is more nuanced than “don’t be evil”.

                      1. 1

                        I certainly agree with that. I still think it’s worth going into, because at a certain point you’re likely to end up doing it anyway. For instance, if we start talking about incentives, we might end up talking about how to incentivize people towards good, or at least, some concept of “not evil”. I’m not saying it trumps incentives or that this is a more effective approach, I’m just saying we should still have the discussion.

                        I think a trap we as engineers often fall into is to attempt to build everything up from laws and axioms. That doesn’t quite work for morality, and the nebulous nature of it means it rarely gets discussed. The software industry in particular is very focused on “solving problems” and never asks questions like “should we solve this problem?”

                        I guess another scary thing about it is that we can’t really empirically verify what the right answer is, and depending on the issue we might even have multiple valid answers. But sometimes just asking the question is worthwhile, even if we don’t have an answer.

                        Perhaps tech companies should start hiring philosophers.

                        1. 2

                          Perhaps tech companies should start hiring philosophers.

                          I’d argue that a good programmer is a philosopher almost by definition.

                          We talk like if our field was an engineering field but most of times we don’t build things constrained by the physical world (yeah I know what latency is… I said most of times :-D).

                          Or we talk like if our field was just applied math, pure and intangible, but then we talk about usability or we kill someone through a self driving car.

                          But ultimately we work with ideas.

                          The choice to ignore the ethics of our work is up to us.

                          But we have much more instruments to think about our role in the world than any “professional philosopher” hired to think for us (in the interest of the company).

                          1. 1

                            if we start talking about incentives, we might end up talking about how to incentivize people towards good, or at least, some concept of “not evil”.

                            That’s how you do it. In Google’s case, a publicly-traded company, that means you have to hit them in the wallet in a way that knocks out the contract. Alternatively, convince their management to change their charter or use other legal means to block whole classes of action in the present and future that they agreed were evil. I’m not sure if that would even work in Google’s case but one can start businesses like that in nonprofit or public benefit form.

                        2. 1

                          I think friendlysock was trying to illustrate the point with some examples. The comment succeeded given the other person understood the points. There’s nothing wrong with that. You said to instead work off claims about evil in this situation based on what people are saying. In this case, what does evil mean exactly to both those employees and various stakeholders in the United States? Based on the political debates, I know there’s quite a few different views on whether these programs are evil or not. Even within the main, political parties, in Silicon Valley, and in Google itself.

                          The only thing sure is that about 4,000 of Google’s 70,000 people plus some other folks writing a letter don’t like what Google is doing. Of the 4,000, only a dozen or so showed it’s worth not working for Google. So, that’s under under 1% of Google’s workforce. The others are continuing to support Google’s success, including that program indirectly, while protesting that program. They may or may not leave but I think most will stay: workers gripe more than they take action in general case, esp if employer’s actions is morally a mix to them or six digits are involved. If they leave, there’s a lot of people willing to take their place with no long term effect on Google. The remainder and some new hires collectively are apathetic to this or believe it’s morally acceptable.

                          Many of the people staying would probably tell you they’re decent people with Google doing a lot of good for the world (arguably true) despite this evil. We saw this in NeverAgain pledge. Others would tell you this kind of thing is inevitable enough that Google not doing it would make no difference. Some of them would even say it’s better if they do it so they can do it right minimizing harm. Yet another group will claim these programs prevent a larger number of deaths than they cause or prevent real damage vs hypothetical risks detractors talk about. People ranging from those developing software to those doing drone strikes might believe they’re saving lives in their work while the dozen that quit will be doing less valuable work in tech for their own benefit.

                          I don’t think there’s a clear answer of evil if I’m looking at the stakeholders in this discussion. They’re all over the place with it. The acting public is in a few camps: those doing a mix of opposing and tolerating drone operations who lost the election; those mostly supporting them whose party is in control; billions of dollars worth of users and businesses who don’t care enough to switch providers; tiny, tiny, tiny slice of revenue from those that will. Put in that light, nothing they’re doing will matter past their own conscience. Hell, those thinking the tech is evil might have been better off staying in there half-assing the programming on purpose to make it look like such tech just isn’t ready to replace people yet. There’s precedents for that with many of them in defense industry except for profit rather than moral reasons.