Pareto optimality, utilitarianism, and killing machines. Stapled of science fiction since the first robot, and probably since all the holy books of all the religions. cf Golems, contracts with devils. But now it the golems are weaponsised 3d-printable downloads
The trolley problem in the age of machine agency, war drones and smart cars. (Also, what is “agency” anyway?) Hell, even if we can design robots to solve ethical dilemmas, do we want to? Do instinctual human ethics have an especially good track record? What are the universals specifically?
popsci podcast Radiolab’s does an introduction to the Trolley problem
Moral machines is a deadpan attempt by MIT to elicit the moral code:
We show you moral dilemmas, where a driverless car must choose the lesser of two evils, such as killing two passengers or five pedestrians. As an outside observer, you judge which outcome you think is more acceptable. You can then see how your responses compare with those of other people.
For humans as cogs in the machine
In a raging flood, a man risks his life to save a swept away child, but two years earlier he voted against strengthening the levee whose breaching caused the flood. During an epidemic people work tirelessly to help the stricken, but ignored elementary sanitation processes that could have prevented the outbreak. More astoundingly, as many as 200,000 Americans die each year from diseases spread by their own doctors who have been ignoring elementary sanitation (including simply washing their hands when needed), but who then work diligently to try to save the patients they have infected. Studies show that about 80% of Americans are “highly concerned” about climate change, yet this percentage drops to less than 20% when the issue is combined with what it will cost to actually deal with these changes. […]
In our world, we have enough power to topple our most important systems, but not the power to restore most of them.
Being heroic in the face of disaster—as humans often are—will not help in most of these cases. This means that we have to “learn about consequences before they happen”. We have to be able to summon vivid enough imaginations of the disasters to be heroic long before they happen.
Imagine you came across a child drowning in a small pond and you were the only one around to help. You could easily save the child by wading in, although doing so would ruin your clothing and shoes. But if you don’t, the child will die.
It’s a no-brainer — you should save the child. Would the answer be any different if there were others around who could also help? No. Should it make any difference if the desperate child wasn’t directly in front of you? No.
The question, then, is are you any less obligated to intervene if the child isn’t drowning but is instead in mortal danger due to lack of food, water, or medical treatment and the only means you have to help is donating money to charity? […]
The moral principle in both cases is the same: we ought to reduce the suffering of others so long as doing so does not require “sacrificing anything nearly as important.” […]
It is this basic argument that has inspired a growing social movement, which brands itself Effective Altruism. Effective Altruists calculate where expendable income is best spent and encourage the relatively affluent to channel their capital accordingly. Among their most highly favored causes are the Against Malaria Foundation (which distributes insecticide-treated bed nets), the Schistosomiasis Control Initiative (which works to establish school-based deworming programs), and GiveDirectly (which gives unconditional cash transfers to people in extreme poverty). […]
The core problem is the bourgeois moral philosophy that the movement rests upon. Effective Altruists abstract from — and thereby exonerate — the social dynamics constitutive of capitalism. The result is a simultaneously flawed moral and structural analysis that aspires to fix the world’s most pressing problems on capital’s terms.
[…] capital’s commodification of necessities directly undermines the self-sufficiency of entire populations by determining how resources are allocated.[…]
In the meantime, capital extracts around $2 trillion annually from “developing countries” through things like illicit financial flows, tax evasion, debt service, and trade policies advantageous to the global capitalist class. […]
These dynamics, which spring from capital’s insistence on the commodification of necessities, are what turn billions of people into drowning strangers and generate a need for ever-multiplying charitable organizations in the first place. […]
Effective Altruists like Singer begin and end their analysis at how to deal with moral dilemmas downstream from these dynamics. This is what makes Effective Altruism particularly pernicious. […]
The problem, apparently, isn’t that capitalism’s institutionalization of immoral maxims ends up leaving billions in poverty and hundreds of millions in existential need of food, water, shelter, and basic medical care.
Instead, the problem becomes that relatively affluent individuals haven’t bought those necessities from the capitalist class for the hundreds of millions that need them; the comparatively wealthy have been “living high and letting die” either out of ignorance of what their money could buy or out of weakness of will in the face of a consumerist society.