We are rapidly approaching a world of self-driving cars. Experts contend that as compared with their human-driven counterparts, these computer-driven machines promise to save vast numbers of human lives. We also know that self-driving cars will inevitably get into accidents, sometimes with deadly consequences. As with human-drivers, self-driving operating systems will occasionally confront tragic choices, choices that ultimately rest on very human programmers.
Analysts often start with variations on classic “trolley problem”: If a trolley car with failed brakes is headed toward another trolley car with large number of passengers that appears stuck on the rails ahead, is it ethical for the controller to to pull a switch, thereby diverting the moving trolley onto a dormant track with the certain result of killing an identified railway worker who had until then been along a closed section? Assuming an equal value on each life, a pure utilitarian easily resolves the problem in favor of the switch, but she does so by assuming the merits of utilitarianism. Other moral values complicate the analysis, as is characteristic of constructed ethical dilemmas implicating the tradeoff between saving a larger number of statistical lives versus knowingly taking a smaller number of certain lives.
In today’s USA Today article, titled Self-driving cars programmed to decide who dies in a crash, Todd Spangler of the Detroit Free Press
updates this conundrum to account for self-driving cars. Here I will take some license with his examples. Imagine, for example, a self-driving car, also with faulty brakes, headed into a crowded sidewalk, uncertain as to who or how many might successfully disperse, yet with the option to swerve, thereby killing one person with certainty? Or imagine the choice between protecting a fully occupied self-driving car’s driver and passengers versus protecting a single pedestrian, or the lone driver in another vehicle? And how might the answers change if the pedestrian, or a passenger in either car, is a child? We obviously could go on and on. Instead, I want to reframe the conversation. I do not claim that this alternative framing solves each ethical dilemma. It does not. Indeed, it likely raises new ones. But at the very least it will help to solve some, while also sharpening other, ethical dilemmas.
The problem is the failure to consider that in an era of changing technology, we need differing default rules than when we are in an era of stable technology. This is easily captured in game theoretical terms. This requires me to introduce one somewhat intricate concept, but please bear with me; I promise that the illustrations that follow will make it quite clear. The concept is known as Nash equilibrium, or sometimes pure-Nash equilibrium. This means that when two or more players confront a formal game, with payoffs associated with each player's combined strategies, there is a single outcome, or a set of outcomes, such that no individual player can improve upon her expected payoff with a unilateral change in strategy.
In the prisoners’ dilemma, for example, the prisoners are put in separate rooms and each is told the following: If both confess, each will be convicted of a moderate offense and serve three years; if one confesses and rats out the other, who remains silent, the rat walks, and the other gets five years; and if both remain silent, each gets convicted of a minor offense and serves six months. Although each player would get a better result, serving only six months, if both remained silent, regardless of the other player’s strategy, with these incentives, each prisoner is rationally motivated to confess and thus to testify against the other as a means of securing a lighter sentence. The consequence is a single pure-Nash equilibrium of mutual defection.
In the classic driving game, constructed well before self-driving cars, if the first driver anticipates that the second will drive on the right side of the road, it is rational to mimic that strategy, also driving right; conversely, if she anticipates the second driver will drive on the left side, it is rational to mimic that strategy, also driving left. Here there are two pure Nash equilibria: both drivers driving left or both driving right. There's no incentive to deviate as doing so will lower the payoff for the party who does (and for the other as well). And yet, with limited information and guess work at the other player’s strategy, it is possible that the drivers end up at a mixed strategy in which one drives left and the other right, potentially with tragic consequences.
The analytical problem that confronts today’s USA Today article, and similar analyses, on the ethical dilemma of self-driving cars arises from failing to recognize the implications of the preceding game theoretical insight. The promise of self-driving cars is greatest if all cars are self-driving. That is because, as experts have pointed out, self-driving cars make very “smart” moves that are nonetheless counter to human driving intuition. Self-driving cars thus reduce risk by making decisions we humans don’t expect them to make. If all cars were self-driving, this would not be a problem at all; self-driving cars will expect other self-driving cars to follow the same strategies they do, just as humans who now drive cars generally expect other drivers to respond as they do. Genuine ethical problems become more acute in a world that combines of self-driving and non-self-driving cars, and that therefore introduces considerable guess work. This combination, and the resulting risk of mixed strategies, raises its own set of profound ethical questions that have not been adequately explored, and those questions differ substantively from the types of pure programming questions implicated in a world with only self-driving cars.
NYU Law Professor and Economist Lewis Kornhauser, wrote an article titled An Economic Perspective on Stare Decisis, that helped frame this problem as it arose in an earlier era. (See especially pp.70-71; see also this reply by Yale Law Professor Jonathan Macey.) Imagine the beginning of automobiles in the early twentieth century. This technology began without much by way of formal rules of the road or licensure requirements. As a result, the then-novel technology gave rise to potential accidents involving pedestrians or others, such as those riding a horse and buggy, who were unaccustomed to sharing the road with driven machines. Should automobile drivers in this era have been strictly liable when, for example, a pedestrian unexpectedly, and tragically, appeared on an unpaved road and was struck? We cannot solve this ethical question by saying that the driver or non-driver was “at fault.” The ethical, and legal, question is precisely about how fault is ascribed. We might also imagine contemporaneous newspaper articles debating whether the burden should be on drivers or pedestrians to take special care. But that debate likewise misses a critical point: the answer depends in large part on which technology society places a higher value on in a period of technological transition. Today, in the early twenty-first century, pedestrians, or others in transit, know to take extra care before entering onto any road; one-hundred years ago this was less obviously true. When pedestrian traffic is more highly valued, the ethical, and also legal, burden should rest with drivers; when vehicular traffic becomes more highly valued, that ethical, and again also legal, burden should likely flip. This is also true respecting self-driving and human-driven cars.
There is no fixed answer to the ethical questions posed above, at least in periods of transition. Instead, the answers depend on the pace of transition involving human-driven versus self-driving vehicles. At some point, self-driving cars will become dominant. I anticipate that some day, however soon, we will face regulatory incentives to rid traffic of non-self-driving cars. Most likely this will begin in major cities, where the risks of accidents arising from combined traffic is greatest, and then work outward to suburban, then rural, communities. As these transitions occur, the value of human-driven cars will fall, and the value of self-driving cars will rise. This will spur greater investment, which will drive down prices and also improve technologies for self-driving cars. Overall, we can anticipate three general periods: (1) human-driven car dominance; (2) a mixed regime with both types of cars; and (3) self-driving car dominance. The problem is that the answers to today’s questions depend on whether we are in periods 1, 2, or 3, and most commentators fail to acknowledge how the existence of these changing periods affects the analysis.
I’ll close with a simple illustration. The linked article poses the question whether a car programmed to save its own passengers over others is ethical, and then asks, assuming it is ethical, whether that changes if the other person, say a pedestrian, is a child. The answer, however, should be contingent on which regime we are in. If we are in 1 or even 2, then it is likely unethical to favor passengers of the novel, and thus outlier, technology when programming a self-driving car (even if they are of greater number) over others who are behaving consistently with the more commonly embraced technology. This result is apt to flip, however, as we enter period 3. We might treat children differently in each case, assuming this is known, and, of course, there will always extreme cases, such as a child somehow wandering on to a multilane limited access highway.
I want to be clear. I do not claim that game theory avoids hard ethical questions. Nor do I claim that game theory alone answers these questions. My claim is more limited. Game theory helps frame these questions, and without a proper framing, we tend to operate on unstated yet problematic assumptions.
I welcome your comments.