From What Happened, p. 241:

Technologists like Elon Musk, Sam Altman, and Bill Gates, and physicists like Stephen Hawking have warned that artificial intelligence could one day pose an existential security threat. Musk has called it “the greatest risk we face as a civilization.” Think about it: Have you ever seen a movie where the machines start thinking for themselves that ends well? Every time I went out to Silicon Valley during the campaign, I came home more alarmed about this. My staff lived in fear that I’d start talking about “the rise of the robots” in some Iowa town hall. Maybe I should have. In any case, policy makers need to keep up with technology as it races ahead, instead of always playing catch-up.

Update 11/24/2017: Clinton said more about AI fears in an interview with Hugh Hewitt:

Bill Gates, Elon Musk, Stephen Hawking, a lot of really smart people are sounding an alarm that we’re not hearing. And their alarm is artificial intelligence is not our friend. It can assist us in many ways if it is properly understood and contained. But we are racing headfirst into a new era of artificial intelligence that is going to have dramatic effects on how we live, how we think, how we relate to each other. You know, what are we going to do when we get driverless cars? It sounds like a great idea. And how many millions of people, truck drivers and parcel delivery people and cab drivers and even Uber drivers, what do we do with the millions of people who will no longer have a job? We are totally unprepared for that. What do we do when we are connected to the internet of things and everything we know and everything we say and everything we write is, you know, recorded somewhere? And it can be manipulated against us. So I, you know, one thing I wanted to do if I had been president was to have a kind of blue ribbon commission with people from all kinds of expertise coming together to say what should America’s policy on artificial intelligence be?

But of course, the worries Gates & Musk & Hawking have expressed are not about self-driving cars.

More Wynne & Udell (2013):

Michael D’Amato and Paul van Sant (1988) trained Cebus apella monkeys to discriminate slides containing people from those that did not. The monkeys readily learned to do this. Then the monkeys were presented with novel slides they had never seen before which contained either scenes with people or similar scenes with no people in them. Here also the monkeys spontaneously classified the majority of slides correctly. So far, so good – clear evidence that the monkeys had not just learned the particular slides they had been trained on but had abstracted a person concept from those slides that they then successfully applied to pictures they had never seen before.

Or had they? D’Amato and van Sant did not stop their analysis simply with the observation that the monkeys had successfully transferred their learning to novel slides – rather they went on to look carefully at the kinds of errors the monkeys had made. Although largely successful with the novel slides, the monkeys made some very puzzling mistakes. For example, one of the person slides that the monkeys had failed to recognize as a picture of a human being had been a head and shoulders portrait – which, to another human, is a classic image of a person. One of the slides that the monkeys had incorrectly classified as containing a human had actually been a shot of a jackal carrying a dead flamingo in its mouth; both the jackal and its prey were also reflected in the water beneath them. What person in her right mind could possible confuse a jackal with a flamingo in its mouth with another human being?

The explanation for both these mistakes is the same: the monkeys had generalized on the basis of the particular features contained in the slides they had been trained with rather than learning the more abstract concept that the experimenters had intended. The head and shoulders portrait of a person lacked the head-torso-arms-legs body shape that had been most common among the images that the monkeys had been trained with, and consequently, they had rejected it as not similar enough to the positive image they were looking for. Similarly, during training, the only slides that had contained flashes of red happened to be those of people. Three of the training slides had contained people wearing a piece of red clothing, whereas none of the nonperson slides had contained the color red. Consequently, when the jackal with prey slide came along during testing, it contained the color red, and so the monkeys classified it as a person slide.

From Wynne & Udell (2013):

Michael Young and colleagues carried out experiments that add to a sense that the pigeon’s perception of pictures of objects is not identical to our own. They trained pigeons to peck in different locations on a computer-controlled touch screen, depending on which of four different objects was presented: an arch, a barrel, a brick, and a triangular wedge (Young et al., 2001). The objects were initially presented to the pigeons as images shaded to suggest light shining on them from one direction. Next, Young and colleagues tested the pigeons with pictures of the same objects, but this time illuminated from a different direction… To the experimenters’ surprise, the pigeons’ ability to recognize the objects was disturbed by changes in lighting that human observers were barely able to perceive… [see below]

Baiocchi & Welser (2010):

…we conducted a literature survey on each of the [problems comparable to the problem of space debris]. We then determined the length of time spent in each stage (problem identification, establishment of normative behaviors, mitigation, and remediation) based on research from periodical sources, legislative records, and court rulings… Finally, we inspected each timeline and made a judgment about the approximate year in which each problem entered a new stage… The result is shown in Figure 6.1, and it provides a notional comparison that shows how each of the problems progressed through the four stages.

Could be interesting to see this kind of analysis for a greater range of societal challenges, or sets of challenges chosen for how similar they are to a different target case. (The target case for this report was space debris.)

The introduction to Jacobson (1984) makes it sound as though the John A. Hartford Foundation was roughly cause-neutral and utilitarian in its approach, at least for some of its history:

The 1958 annual report of the Hartford Foundation describes its starting point:

Neither John Hartford nor his brother George, in their bequests to the organization, expressed any wish as to how the funds they provided should be used… Our benefactors’ one common request was that the Foundation strive always to do the greatest good for the greatest number.

…If available funds are to be used effectively, it is necessary to carve from the whole vast spectrum of human needs one small band that the heart and mind together tell you is the area in which you can make your best contribution.

The first task of the Foundation was thus to define the greatest good. Basing its decision on the pattern of John Hartford’s previous giving, the Foundation chose to support biomedical, largely clinical, research. Between 1954 and 1979, the Hartford Foundation participated in some of the most important advances in modern medicine, supplied hospitals and medical centers with equipment that reflected those advances, provided for the training of a generation of researchers, saved countless lives, and involved itself deeply in the burgeoning of the current health care crisis. In that period, the Foundation spent close to $175 million [presumably this is 1984 dollars, i.e. $408 million in 2016 dollars].

…Many modern research-supporting institutions have chosen to bear the costs of close supervision and peer review in order to ensure the quality of projects supported either directly or indirectly by the public. But both the trustees and the staff of the Hartford Foundation came from a background that stressed minimizing administrative costs so as to maximize benefits to the public. During the Foundation’s first seven years as a leading source of funds for biomedical research, the full-time staff consisted of one person. To achieve quality control at low cost, the Foundation adopted a policy of hiring consultants as they were needed to review particular grant applications.

As a matter of policy, too, the Foundation tried to fund projects and types of research that could not obtain funding from other sources. For example, the Hartford Foundation was the first to pay for the patient-bed costs of clinical research. Filling this gap was clearly desirable. But the Foundation also supported some researchers whose theories or personalities inspired skepticism in their colleagues. These grants were calculated risks. Many of the projects thus supported were unsuccessful; a few have produced major advances in clinical medicine.

When these successes occurred, the Hartford Foundation could have chosen to publicize its role in them. But John and George Hartford disliked publicity. The trustees and staff made this family trait a matter of policy. They believed that being in the public eye was tasteless, a waste of time, and likely to produce an excess of grant requests unmanageable by a small staff. As a result, the pool of grant applicants was limited largely to those who heard about the Foundation by word of mouth — from past grantees or consultants.

Probably the truth is more complicated; I haven’t investigated the foundation’s history closely. Note also that the foundation seems to have cared a lot about the overhead ratio, whereas today’s effective altruists tend to think overhead ratio considerations should be subordinate to impact per dollar.

Have any of my readers heard of any other charitable foundations aspiring to be (roughly) cause-neutral and utilitarian in their approach?