1. 20
  1. 35

    Imagine thinking that at the world’s largest surveillance company that the ‘ethical AI’ team could be anything other than lip service.

    “Hey Ethical AI team, how can we capture all this data from everyone and feed it into an AI system to help scumbags sell things like payday loans to people about to lose their homes?”

    Any response other than “shove it up your hole” is unethical by nature.

    1. 3

      under what other conditions is it possible to do this sort of research?

      1. 15

        a more even distribution of wealth so that normal, everyday people have access to do research if they want to

        1. 3

          ok, I’m not sure that’s a particularly realistic thing to expect Timnit to do before she does her research.

          1. 13

            I’m not criticising Timnit. I’m criticising our entire society. I want people like Timnit to thrive instead of being forced to exist in a system that is fundamentally opposed to the work that she is doing.

            1. 8

              OP’s comment is essentially shifting the burden to the individual, who has a comparatively tiny amount of power. What’s being posed is that a tech behemoth says “we have our own oversight group”, and a person who has the capability to serve on that oversight group, who thinks that such oversight existing is very important, should not serve on that group since it is assumed to be a farce:

              Any response other than “shove it up your hole” is unethical by nature.

              Consider an individual’s actual position in this situation: an individual with the capability to serve on such a group and an interest in seeing that group’s mission fulfilled. If this is important to them, they can either participate in this way, or participate in some other way, or not participate. What other opportunities do they have to participate in such oversight?

              Hypothetically, if someone was in a position where they could pick between being on Google’s AI Ethics team or some greater opportunity to effect change (such as being the head of a government oversight committee and lab with millions of dollars in funding), it would be very stupid to pick the former. The much more likely situation is that the opportunity was to be on Google’s AI Ethics team or something else with significantly less likelihood of being influential.

              OP’s comment only holds if you believe that Timnit was naive going into the relationship, or that she had significantly better opportunities to effect change that she turned down. The idea that Timnit would be simply naive going into that position is, I think, an in-kind criticism.

              1. 4

                OP’s comment is essentially shifting the burden to the individual, who has a comparatively tiny amount of power.

                You are not OP and you do not speak for me. Don’t put words in my mouth. Never once did I mention the researcher, nor their role. I talked about the oxymoron of the concept of an ‘ethical AI team’ in an organisation founded on unprecedented levels of surveillance, providing an example of unethical actions the broader organisation enables. My comment doesn’t shift the burden to the individual, it’s about an organisation inherently corrupted by the breadth and depth of it’s actions.

                1. 2

                  imagine thinking that other people lacked the agency to interpret the things one says.

    2. 34

      I find the title on this to be very strange. The paper did absolutely nothing. People did things.

      1. 19

        I find it strange that statements like “training this model had a CO2 footprint of 5 cars over their life time” are not put into more context. How often do they train a model? How many people does it serve?

        Conceivably, the researchers working on it, also had cars which might exceed this carbon impact already . 5 cars for a model of huge impact world - wide doesn’t necessarily seem like a lot.

        Edit :possibly the paper does have more context, of course

        1. 7

          Agreed. It’s interesting to note the exponential growth, though. Their 2017 model 27kWh - less than a single US gallon of gasoline. Also note that BERT and its derivatives have really captured imaginations world-wide, and the approach people use seems to be to throw more data and processing power at it. It’s not just Google doing this, it’s dozens of dumb startups.

          The problem with using judging any activity by its carbon emissions is that we’re likely to need ALL available energy, fossil or renewable, for the purpose of transitioning to a fossil-free economy by 2050, if we want to have a shot at RCP2.6. In that light, any economic activity - whether it’s training a neural network or selling hot-dogs - that’s not aimed at reducing carbon emissions, is somewhat unethical.

          1. 11

            Not to get too political, but our society seems extremely inept at solving pretty much any problem of any worth, whether that’s climate change, Google/Facebook knowing all sorts of things about you, all sorts of muckery with food (varying from palm oil being in damn near everything to food being flow in from the other side of the world), sweat shops in Bangladesh and similar countries making our clothes, etc. etc. etc. Most polls show a vast majority of people don’t like any of these things, but … nothing happens, in almost any country.

            In short, pointing fingers at Google and such with “you should not do that” is probably the wrong strategy, and instead it might be smarter to reconsider how we deal with these problems in the first place. I have some ideas about this, but I won’t expand on them here. I also think it’s exceedingly unlikely that this will happen anyway, for various reasons, so 🤷‍♂️

            tl;dr: we’re fucked anyway.

            1. 8

              Not to get too political, but our society seems extremely inept at solving pretty much any problem of any worth, […] Not to get too political, but our society seems extremely inept at solving pretty much any problem of any worth,

              The problem is that as long as the people with purchasing power do not feel the pain, we are happy to pay lip service to such causes, but do not drastically want to alter our way of living to solve these issues. However, if it is something that affects rich nations, then suddenly a lot is possible. E.g. see SARS-COV-2 vaccines. Western governments have thrown billions at it and within a year it’s done (of course, based on prior work on SARS and MERS).

              Of course, climate change affects us all, but rich nations do not really see it yet, with some exceptions (fires in Australia and the US West Coast). Of course, climate change is too hard to turn around to do something when we really start caring.

              1. 6

                Yes. What I was trying to argue was that while saying it’s unethical to spend 600 MWh on a language model is completely true, it’s not particularly insightful, as it’s unethical to spend 600 MWh on almost anything - including those six cars that the previous commenter dismissed as a trifle.

                I actually find that this type of argument - a new technology being unethical because of its embodied energy - understates the actual shape and size of the problem. A lot of our current technological infrastructure is ridiculous, when measured by that same yardstick. But maybe it’s unfair to judge a paper by its editorialized summary.

                1. 1

                  Google/Facebook knowing all sorts of things about you

                  Most polls show a vast majority of people don’t like any of these things, but … nothing happens, in almost any country.

                  Is GDPR not a solution? After all, the reason Google knows so much at this point isn’t really search, it’s the octopus of an advertising business it’s got.

                  1. 4

                    GDPR essentially says you can continue doing what you did before as long as you ask consent: so, you get popups, and other than that little really changed. The exact interpretation of various things (such as “implied consent” if you never click “accept”) differs per member state, and there’s also the issue of enforcement which is up to member states. In short, it’s all pretty patchy. And lot of these popups are designed in such a way that opting-out is quite time-consuming (not necessarily on purpose, could just be crap design).

                    In the end, I feel GDPR is perhaps well-intentioned, but it’s also designed so that companies can keep doing what they were doing while offering an “opt-out solution”, which in many causes is a faux-solution. If something is widely considered undesirable then it should not be done at all, instead of relying on savvy-enough consumers hunt for opt-out mechanisms.

                    A lot of the other things, such as “right to access your data” and “right to have your data removed” were already part of the laws in many countries before GDPR, but no paid much attention to that because who cares what the laws are in some tinpot little European country, right?

                  2. 1

                    food being flow in from the other side of the world

                    Very little food is moved via air freight, no? Doesn’t a massive, massive majority of food transport consist of rail, truck and container ship (and combinations thereof)?

                    And I hesitate to ask this, because supply chain logistics feels a bit off-topic for lobsters, but what about that is “muckery” anyway?

                    1. 3

                      With “muckery” I meant more like sugar being added to a lot of stuff, lemons being coated with wax to make them look nicer in the supermarkets (but it’s not good if you use lemon zest), trans-fats not being very healthy in spite of being marketed as such and industry pretending it’s not a problem, I could go on and on.

                      As for logistics, shipping is never really free, especially since a lot of foodstuff are cooled. When I lived in New Zealand things were much more seasonal (you can buy imported tomatoes out-of-season, but you pay ridiculous prices).

                      1. 2

                        Avocados were infamous for being transported on planes (to Europe and Asia, at least), but I see that they’ve moved on to refrigerated sea containers.

                  3. 4

                    The entire thing reeks of performative theatre.

                    1. 2

                      5 cars for a model of huge impact world - wide doesn’t necessarily seem like a lot.

                      One of such example is the models they use to drive the PUE of their data centers to a record low value.

                      1. 2

                        I agree with the alleged point that the cost of pretraining new language models puts it out of the reach of most researchers, resulting in a certain amount of unfair competition. If I have an idea for a better training objective, I cannot really put it to test, because I don’t have the resources, while Google could do easily do a grid search. I don’t think we have seens such a large imbalance in computational linguistics research before.

                        However, most scientific papers that use large transformer networks do not pretrain transformers, but finetune them for a specific task, which typically only takes a few hours on a consumer-level GPU. So, even though the carbon dioxide cost of pretraining a large transformer may be very large, the ‘amortized’ cost is relatively low. Once Google, Facebook, et al. release a new pretrained models, thousands of models (if not more) models are created by finetuning the pretrained transformer. So, the CO2 impact per actual per published/deployed model is probably not that much higher than before pretrained transformers.

                        The point about biases in models and their ramifications for society are on the mark. I fear that the author’s points are not compatible with Google’s PR about these models and we should hold FAANG and others accountable for such issues.

                        1. 1

                          My shallow understanding of online AI products gives me the idea this is a step in a periodic CI/CD pipeline. Think nightly builds, but the inputs are both code and training data. I think if only the data changed you could just refine the previous result, but in the case of code changes (capturing a new kind of information, any changes to the network structure) you would have to start over and train from zero. This is just a remote guess; I’d love for someone who knows to speak up.

                        2. 27

                          Flagged, for a few reasons:

                          • Google HR decisions (misguided or not) don’t really belong here.
                          • The article isn’t the actual paper itself, but a summary by a third party.
                          • The title isn’t accurate–the content of the paper (again, which we haven’t seen) seems to be less important than a list of demands required to stay at GOOG.
                          • This is covered elsewhere in the news.

                          This is bait, folks. Please don’t add tires to the tirefire.

                          1. 2

                            a list of demands required to stay at GOOG.

                            That links to https://www.platformer.news/p/the-withering-email-that-got-an-ethical which quotes one of her emails directed at her team which states among other things: “What I want to say is stop writing your documents because it doesn’t make a difference.”

                            I’m not sure if any manager gets to stay in their position if they tell people, some of whom report to them, on the record that their assigned work is futile and they probably shouldn’t do it.

                          2. 10

                            Google AI was the first to invent the Transformer language model in 2017 that serves as the basis for the company’s later model BERT, and OpenAI’s GPT-2 and GPT-3. BERT, as noted above, now also powers Google search, the company’s cash cow.

                            You know, I never thought about this before, but Google must have gone through a lot of different backends to their search service. I remember learning about the page rank algorithm in Linear Algebra, and the professor remarking about how it was the first way of doing search ordering when Google was first founded. I’m sure there have been several algorithms in between the original pagerank matrix math and the current BERT backend. Is the history of that publicly accessible? Have some older ML models had tenure in that office?

                            1. 6

                              I remember having to code an implementation of Page Rank in OCaml in an algorithms class. I found it funny that “Page Rank” is named after Larry Page, not because it ranks web pages…

                              1. 5

                                Nominative determinism at work.

                                1. 3

                                  I believe it was meant as a pun even at the time.

                              2. 3

                                … the six collaborators drew on a wide breadth of scholarship. The paper’s citation list, with 128 references, is notably long.

                                Honest question since I know next to nothing regarding research papers: does the amount of references a paper has matter to people who generally tend to read them? If so, why? Do other papers in the field tend to have less?

                                1. 12

                                  It’s notable here, as one of the arguments Google’s given as defense of ordering the paper retracted is that it ignored too much other relevant research. 128 references is about an order of magnitude higher than what I feel is typical, based on the papers I’ve read. To be fair, I haven’t read many papers on the state of research and you’d expect that kind of review to have more references, but this absolutely smashes any expectations I’d have and makes the claim that it ignored research hard to swallow.

                                  In general, the quantity of good references in a paper reflects how well it’s grounded in existing research. How important prior work is to the paper’s claims will depend heavily on the paper in question. And, of course, if the citations are all garbage then it’s a huge red flag, but I haven’t heard anyone claim that here.

                                  1. 6

                                    I think it depends on the type of paper. If it’s a paper presenting brand new research, you’d expect to have a few references (my personal, very rough metric is “about as many as there are pages in the paper,” although they don’t have to be evenly distributed in terms of citations, and isn’t a hard and fast rule!); if it’s a survey paper or something that is collecting lots of other research and commenting on it in some systematic manner then I would expect more.

                                    128 seems a lot by most measurements, but if it only had 30 references (and interesting subject matter) I probably wouldn’t throw it out. I don’t think it’s overly important.

                                    1. 4

                                      I would imagine it matters in peer review. I have heard of cases of papers being rejected because they don’t cite some paper the reviewer wrote. Citing a lot of papers could also be some sort of defence against that. Not saying that’s what this paper is doing though.

                                    2. 2

                                      From Timnit Gebru’s Twitter posts:

                                      I said here are the conditions. If you can meet them great I’ll take my name off this paper, if not then I can work on a last date. Then she sent an email to my direct reports saying she has accepted my resignation. So that is google for you folks. You saw it happen right here.

                                      “Either heed me or I’ll end my employment”, and the employment was ended. Who was forced anywhere?

                                      If you give an ultimatum, perhaps you should be ready for it to backfire.

                                      1. 3

                                        If Google had taken the last date option, we probably wouldn’t be having this conversation. An immediate firing indicates that they were unwilling to deal in good faith - perhaps because they justifiably doubted her own good faith, sure, I don’t have enough information to be certain. But if I were an employer, I’d expect my employees to be honest about the conditions they were willing to work with, and prioritize a smooth exit whenever possible.

                                      2. 1

                                        It’s hard to tell if the research paper had good points because it constantly felt that the author was injecting their own opinions. Wasn’t expecting so many parenthesis’d comments.

                                        Admittently, the thought of energy efficiency on training these large models seems relevant, but if there’s no discussion on how much that usage is amortized across the training’s lifetime (train intensely, use extensively), it seems like there’s more to the picture.