The cold equations of ethics
Cory Doctorow has written a thoughtful critique of two science fiction stories and how they might promote short-sightedness and morally bad behaviour. If one thinks science fiction is good for teaching us to think about the future (or literature in general about the world), is there a moral responsibility of authors to avoid moral hazard? And if that is true, what about ethicists coming up with thought experiments?
The first story critiqued is Tom Godwin’s classic 1954 story The Cold Equations. In the story a situation arises where the only moral choice is to throw an innocent girl out of the airlock. All other options lead to worse outcomes. The intended moral is that human values has no chance against the constraints imposed by the cold equations of physics: sometimes reality dictates what you must do.
Except, as Cory notes, the reason reality dictates it is because the author dictates it. Godwin worked hard to constrain the story such that throwing the girl out the airlock was the only option, despite several attempts by the editor to propose a solution. Background assumptions are contrived to ensure the outcome. She died because the author wanted it. This made the story poignant and emotionally wrenching – it is discussed to this day, after all – but it doesn’t say anything morally relevant. If there is no choice, there is no moral action occurring: it only looks that way. This, Cory argues, is a moral hazard that encourages people to behave badly in constrained situations.
By imposing the right boundary conditions an author can make even extreme behaviour moral. This makes for good stories, but they are not teaching us how to think well about the future since they depend on a contrived situation. Maybe such contrived situations could occur, but real situations have far less coherent contexts and hence make the moral issues far more non-trivial. Since well written stories are salient and memorable, we often use them as examples when reasoning about the real world… despite the risk of their conclusions being arbitrarily prescribed.
My own favourite example of this kind of induction of conclusions is Daniel Keyes’ Flowers for Algernon, which is often used as an argument that cognitive enhancement will not be good for people. However, the reason the protagonist suffers is that he and the world is written that way. While it makes for a good story it doesn’t help us think much about the nature of intelligence enhancement despite apparently giving us an argument.
This suggests that writers trying to be moral should do their best to refrain from overly constraining their fictional worlds in order to tell stories that actually help us think about acting morally in the real world.
However, many authors have moral views they wish to promote through their art. This might lead to a trade-off between writing a compelling story that makes people more likely to behave in a more moral manner, or make a less compelling story that makes people think more widely about the moral issue. Should they teach morality or ethics? The answer is probably individual: some people are moralists, other have a more ethical bent. There are examples of good and bad fiction of both kinds.
Philosophical thought experiments can be criticised similarly. In the basic switch-case trolley problem there are only two options, to switch or not. It is a deliberately constrained situation, aiming at teasing out some moral intuitions or show where deontology and consequentialism diverge. It is always amusing to introduce it to engineering students, who reliably try to take the third option (jam the switch into the wheel!) rather than get on with the program. But the extreme constraints of course make the experiment so remote from reality that it might not say much about real moral choice. It might be a useful tool for teaching formal ethical reasoning but it might not help with everyday ethics. And people might come to mistaken conclusions about how they should act because they hear ethicists use it. The ticking bomb scenario might similarly have its place in an academic setting, but 24 is teaching a generation what the “right” answer is – partially by postulating that torture does work.
Taking the third option is one area where science fiction often excels, from Captain Kirk’s resolution of the Kobayashi Maru scenario to Arthur C. Clarke’s brilliantly simple but surprising story Maelstrom II. It is the victory of intelligence and creativity over constraint. Third option stories evade the false moral dilemma or required solution imposed by authorial constraints. A good author will make the situation as contrived as necessary but the solution non-obvious yet not too contrived (a deus ex machina does not make for a good ending). Of course, sometimes this evasion is not a good thing: the Star Trek II film actually makes the point that evading the scenario might be ingenious but misses an opportunity for personal growth. Yet overall stories taking the third option allow agency and moral responsibility in a way that stories where the outcome is prescribed does not.
Does this mean that we should hope our students find a way for Jim to save the Indians, work together with the Fat Man to stop the trolley, or suggest an ingenious surgical procedure to save Thomson’s violinist? Like the author trying to promote a particular moral view, a good philosophy tutor tries to promote good ethical thinking – the ability to reason about abstract moral principles. But good ethical thinking is not the same as good moral thinking – trying to find a way of acting that solves moral problems, sometimes by preventing them in the first place. So just like the author case there might be a balance needed between the super-constrained cases that allow clean reasoning about principles, and the less constrained cases that actually look like practical ethics.