1. 4
  1.  

  2. 9

    Not the first article to make this observation, and not the last. IMO it’s not as interesting a thought experiment as the moral dilemma of the train switch that kills 10 without action or 1 with action.

    The sad answer is this is just an insurance decision. The better answer is that while there will be atrocities, there will be far less than those committed by idiots who are drunk or texting or unskilled.

    1. 7

      “No human of moderate intelligence and education would have made that mistake. No such human could understand another such human making that mistake. Indeed, any human who insisted on that answer, as Watson surely would have, might very well be deemed legally insane.” <- I don’t think this is actually true. It’s entirely within my realm of experience that a human of moderate, or even high, intelligence and education might make one individually-stupid claim, like that Toronto is a US city. Especially in a high-stress situation where one is expected to give an answer very quickly. I could absolutely see a human being, even one who is smart enough to do well on Jeopardy, giving “Toronto” for that answer (and then feeling stupid about it immediately thereafter)

      1. 6

        Wow oh wow do I hate this argument.

        “Driverless cars will KILL PEOPLE because COMPUTERS AREN’T HUMAN!”

        No sh* sherlock.

        They positively WILL kill people, but the numbers will be STAGGERINGLY smaller than the number of people who die every single day because humans are total dumb asses and are paying more attention to their smart phones than the road. I live in a city. I take the bus - I see it all … the… time…

        1. 5

          I had no idea how bad it was until I rented a moving truck and drove 1500 miles. I could see into cars where normally I cannot.

          I would estimate that every 20 cars, one driver is on their phone. It is crazy bad.

          1. 2

            But how do you know that? I am constantly astounded by how often people seem to be dramatically underestimating the difficulty of creating reliable computer vision and software that can handle adverse driving conditions based on nothing but Google press releases.

            1. 3

              I don’t under-estimate the difficulty at all. It’s a problem that’s being worked on and iterated against. Given enough iterations, I suspect driverless cars will become eminently usable and reasonably safe.

              Also, before you tell me why I shouldn’t want this, please bear in mind that I’m partially blind with fine and gross motor impairment - I will never be able to drive, and for a huge chunk of my life this was a very big problem.

              So yes, when the time comes, I will do my homework, I will do a proper risk/reward evaluation, and if it’s favorable I will do the touchdown dance, because I have been waiting for the ability to be transportationally independent my entire life.

              1. 4

                Same. Although there’s some hope that I may medically be able to drive in the future, currently I’m not, and it was an enormous problem for me for most of my life. It creates a huge power dynamic for any abuser to exploit, and even when that’s gone, it makes simple tasks dramatically more time-consuming and expensive. I’ve been waiting very impatiently for self-driving cars to be commercially available.

                It’s great that people come up with inspecific, idealistic proposals like “society should do something”, but I’ve found that most people saying that aren’t interested in owning those proposals and making them happen. At the very least, as a disabled person, I ask that people who want there to be non-technological solutions for disability issues not oppose the technological efforts.

                1. 6

                  I’ve found that most people saying that aren’t interested in owning those proposals and making them happen.

                  Quite a few of us who don’t drive do put quite a bit of effort into the political side of things, partly because I believe this is the only really workable means of transportation in livable communities (i.e. not the kind bisected by highways), and well, if for no other reason, because I exclusively get around by transit and would therefore like it to be better. It’s slow going of course, but I try to get involved in both local transit-advocacy groups and larger-level political campaigns whenever I can, and some things have been improving, although too much of the money does still go to flashy projects like high-speed rail that don’t do a lot for everyday mobility.

                  I ask that people who want there to be non-technological solutions for disability issues not oppose the technological efforts.

                  That’s fair. I think many of us would also ask the flip side of this, that proponents of technological efforts should at least not oppose better transit, maybe ideally even support it as one component of improved mobility. When I lived in the Bay Area at least, affluent techies seemed extremely negative on transit, viewing the bus (god forbid) as some kind of backwards 20th-century socialist thing that only the poors use, contrasted with the libertarian utopia that Elon and his self-driving Teslas will bring us.

                  1. 5

                    I completely agree with all of that, and thank you for adding it.

                    Both mass transit and individual cars are important prongs of mobility, and always will be, though of course we can hope to tip the balance away from cars.

                    1. 3

                      Man, oh man, I drank this kool-aid for so long I own stock.

                      Living a public transit only life is maybe a viable thing if you live in NYC, or if you live in Boston and are willing to restrict yourself 100% to the goods and services you can get to in the city proper, but a very substantial portion of people who mainly rely on public transit in Boston also say “and hey I can rent a ZipCar whenever I want”.

                      And it all falls down right there.

                      I am hugely in favor of better public transit. However I am now 49 years old, and for over 30 of those 49 years, I have watched the MBTA struggle and flail with budget problems, corruption, and of late such serious mechanical problems that we have trains on fire and the red line, my beloved mode of transit that was rock solid 20 years ago, now has minor, moderate or even serious delays just about every single day.

                      I welcome any suggestions for remedying this situation, but at this point I hope you’ll forgive me for perhaps despairing of a brighter public transit future.

                    2. 3

                      Totally agree. It’s super easy for people to armchair quarterback the problem when they’ve driven all their lives. As to society providing a different, better solution, like what, public transit? I’ve been watching the US’s public transit systems fall to pieces my entire life (save perhaps for NYC.

                      1. 3

                        It’s great that people come up with inspecific, idealistic proposals like “society should do something”, but I’ve found that most people saying that aren’t interested in owning those proposals and making them happen.

                        I ask that people who want there to be non-technological solutions for disability issues not oppose the technological efforts.

                        I see this sort of argument so often in my country (the US) and am very frustrated by it. We have a situation where for-profit corporations are putting massive investment into something which could, incidental to their profit motive that is mandated by law, benefit people. People who oppose this development based on serious moral concerns–in this case, perhaps based on how in recent history we have held companies responsible for making negligent decisions that led to peoples’ deaths (when a person does this we call it killing someone)–are shouted down by people who argue that there is some potential benefit.

                        The exact same pattern of argument was used to justify deploying things like the British web blocking system, which of course first was supposed to only block child porn, then became a system for blocking porn, and is now becoming a system for blocking “harmful” information like sex ed or information about feminism and homosexuality.

                        The fact is that some solutions would be great in an ideal world but would almost certainly be very very bad in our current world. Blocking child porn would be great if the same infrastructure couldn’t be abused to block politics, and if governments didn’t abuse the infrastructure to do just that. Self-driving cars would be a wonderful solution if we had an actual system for keeping those who would benefit most from their deployment responsible.

                        Finally, large-scale investment in self-driving cars is precisely large-scale investment that could have been put into developing public infrastructure or for lobbying the government. Again, society should do something, but it should not be the large-scale deployment of self-driving cars.

                        1. 4

                          I appreciate your expressing that. It had not occurred to me that anyone would see those issues as connected.

                          I don’t immediately have an answer for you; certainly I believe that there needs to be a regulatory framework of some sort for self-driving cars, but I understand that not everybody does.

                          1. 2

                            It’s a good argument but, respectfully, I don’t buy it.

                            Child porn is patently harmful to the children it victimizes, and arguably harmful to its consumers.

                            Well regulated self driving cars which have demonstrated an excellent safety record could save lives, not degrade them.

                            Are we there yet? No we’re not, but that doesn’t mean I have to like it when everyone shouts down the idea simply because they find the idea unnerving or scary.

                            1. 1

                              I appreciate you reading the response, and I can see where you are coming from. But why do self driving car proponents have to continue to attempt to trivialize legitimate concerns by saying they are just people “shouting down the idea” as “unnerving or scary”?

                              It’s tautological that self driving cars that save lives would save lives, just like it’s tautological that if someone somehow magically invented child porn that didn’t harm anyone and was a social good then it would be a lot trickier to argue it should be illegal.

                              But once again, that does not say anything about who is going to be held responsible when someone cheaps out and buys a self-driving car (made at low cost and high profit to some billionaire exec) that then kills my mom. How many people in the US went to the jail for the financial crisis, exactly, or for dumping toxic waste that killed innocent children whose parents just wanted an affordable place to live? How many people did those engineers, managers, and executives kill because although they’d probably have preferred not to, hey did all that was legally required and anything further than that would have cost more than the penalties? They went home wealthy, and I can straight up tell you that one is much more than the other.

                              I believe self-driving cars have the potential to save lives. I don’t believe they will be well-regulated, and even if well-regulated under our current legal system, I don’t believe the victims of self-driving cars will find justice unless we fundamentally change our legal system so that killing by corporate negligence has the same human penalty as killing by negligence.

                              Moreover there are better solutions that stand to actually benefit low-income people: how many billions have already been spent on self-driving cars that could have paid for subway subsidies or expansions? Public transportation is a lot less cool, but free ubiquitous public transportation is both demonstratively feasible and immensely socially beneficial.

                              Anyways, that’s all I have to say.

                              1. 2

                                You’re right, it is a tautology. Thanks for pointing that out.

                                Just to be clear, I am not at all opposed to public transit!

                                I would love it if public transit could become so widespread and ubiquitous that it actually gave people a similar level of mobility and freedom that cars give people today.

                                Seriously, that would be pretty close to Utopian in my book.

                                I just don’t see it happening in my lifetime.

                                Thank you for your thoughtful responses and courteous tone. This is somewhat of a hot button for me. Most drivers are very quick to disregard the needs of others because they’ve never been in that situation themselves. Clearly you’ve given the whole thing way more consideration than that.

                                1. 2

                                  Thank you too for being thoughtful and courteous! I’m ashamed to say I hadn’t thought about how self driving cars might help people who are unable to drive regular cars, I can see how that would be a major improvement. Really appreciate you taking the time to reply, I have stuff to think about now. Lobsters is cool!

                        2. 2

                          So yes, when the time comes, I will do my homework, I will do a proper risk/reward evaluation, and if it’s favorable I will do the touchdown dance, because I have been waiting for the ability to be transportationally independent my entire life.

                          With a self driving car you don’t risk only your own life-you risk the lives of others. Our society should provide a better solution for you that is not a self driving car.

                    3. 5

                      I find this to be a very poor argument. In this over simplified case we will have an answer (“Toronto”). If it was a human who made the mistake we wouldn’t even have that. Also we can then make improvements to the AI to prevent “Toronto” to prevent another occurrence.

                      Overall AI doesn’t have to do a task perfectly to be preferable. It only has to be better then the average human.

                      1. 1

                        It only has to be better then the average human.

                        No. If a human screws up and kills someone, they are likely to go to jail. If a robotic car kills someone, the car won’t go to jail, the people who made billions from the cars construction will not go to jail, the engineers who wrote the awful software will not go to jail, and the person who was irresponsible enough to use a self driving car will not go to jail. It had better be perfect.

                      2. 5

                        To quote Humans Need Not Apply by CGP Grey [1],

                        “They [autonomous cars] don’t need to be perfect, they just need to be better than us. Humans drivers, by the way, kill 40,000 people a year with cars just in the United States.”

                        I find the source article’s kind of fear mongering to be mendacious. Axiomatically, plenty of people, including those amongst us whom are highly educated, get simple trivia questions wrong all the time, either by having learned an incorrect factoid, or simply making an error of deduction as they work toward a conclusion.

                        Yes, I am quite confident there will be accidents and deaths attributable to errors in programming / logic / whatever new circuitry is used but the crucial thing to know is that we’re already suffering avoidable casualties. The difference is that as self-driving cars evolve and engineering slowly but surely works out the kinks, progress will be made.

                        CGP Grey continues,

                        “Given that self-driving cars don’t blink, don’t text while driving, don’t get sleepy or stupid, it’s easy to see them being better than humans because they already are.”

                        With all credit given to improvements in campaigns against recklessness and carelessness, the same cannot be said for people falling asleep at the wheel, driving drunk, or driving distractedly. Items such as those appear to be intractable problems.

                        [1] https://www.youtube.com/watch?v=7Pq-S557XQU

                        1. 2

                          So, after something like this, what’s the usual turnaround time for the OpenBSD folks to close these sorts of bugs?

                          1. 3

                            Usually a day or so? “Oh, this code tries to predict what the user wants. That’s dumb. Better to do something simple and predictable.” Not sure that entirely solves the problem as presented.

                          2. 1

                            Machine intelligence is going to behave differently from human intelligence, and thus is going to exhibit different failure modes. This isn’t surprising. Because machine intelligences are both complicated an relatively new*, these failure may take on unexpected dimensions (which may be a surprise in the specific case, but is not itself surprising).

                            *The only thing new is the scale at which they operate, thanks to being able to throw computational power at the problem