Richard Clarke and R.P. Eddy recently published Warnings, a book in which they try to identify “those rare people who… have accurate visions of looming disasters.” The opening chapter explains the aims of the book:

…this book will seek to answer these questions: How can we detect a real Cassandra among the myriad of pundits? What methods, if any, can be employed to better identify and listen to these prophetic warnings? Is there perhaps a way to distill the direst predictions from the surrounding noise and focus our attention on them?

…As we proceeded through these Cassandra Event case studies in a variety of different fields, we began to notice common threads: characteristics of the Cassandras, of their audiences, and of the issues that, when applied to a modern controversial prediction of disaster, might suggest that we are seeing someone warning of a future Cassandra Event. By identifying those common elements and synthesizing them into a methodology, we create what we call our Cassandra Coefficient, a score that suggests to us the likelihood that an individual is indeed a Cassandra whose warning is likely accurate, but is at risk of being ignored.

Having established this process for developing a Cassandra Coefficient based on past Cassandra Events, we next listen for today’s Cassandras. Who now among us may be accurately warning us of something we are ignoring, perhaps at our own peril?

Of the risks covered in the book, Clarke says he’s most worried about sea level rise, and Eddy says he’s most worried about superintelligence.

Below is a sampling of what they say in the chapter on risks from advanced AI systems. Note that I’m merely quoting from their take, not necessarily agreeing with it. (Indeed, there are significant parts I disagree with.)

[Read more…]

From What Happened, p. 241:

Technologists like Elon Musk, Sam Altman, and Bill Gates, and physicists like Stephen Hawking have warned that artificial intelligence could one day pose an existential security threat. Musk has called it “the greatest risk we face as a civilization.” Think about it: Have you ever seen a movie where the machines start thinking for themselves that ends well? Every time I went out to Silicon Valley during the campaign, I came home more alarmed about this. My staff lived in fear that I’d start talking about “the rise of the robots” in some Iowa town hall. Maybe I should have. In any case, policy makers need to keep up with technology as it races ahead, instead of always playing catch-up.

Update 11/24/2017: Clinton said more about AI fears in an interview with Hugh Hewitt:

Bill Gates, Elon Musk, Stephen Hawking, a lot of really smart people are sounding an alarm that we’re not hearing. And their alarm is artificial intelligence is not our friend. It can assist us in many ways if it is properly understood and contained. But we are racing headfirst into a new era of artificial intelligence that is going to have dramatic effects on how we live, how we think, how we relate to each other. You know, what are we going to do when we get driverless cars? It sounds like a great idea. And how many millions of people, truck drivers and parcel delivery people and cab drivers and even Uber drivers, what do we do with the millions of people who will no longer have a job? We are totally unprepared for that. What do we do when we are connected to the internet of things and everything we know and everything we say and everything we write is, you know, recorded somewhere? And it can be manipulated against us. So I, you know, one thing I wanted to do if I had been president was to have a kind of blue ribbon commission with people from all kinds of expertise coming together to say what should America’s policy on artificial intelligence be?

But of course, the worries Gates & Musk & Hawking have expressed are not about self-driving cars.

More Wynne & Udell (2013):

Michael D’Amato and Paul van Sant (1988) trained Cebus apella monkeys to discriminate slides containing people from those that did not. The monkeys readily learned to do this. Then the monkeys were presented with novel slides they had never seen before which contained either scenes with people or similar scenes with no people in them. Here also the monkeys spontaneously classified the majority of slides correctly. So far, so good – clear evidence that the monkeys had not just learned the particular slides they had been trained on but had abstracted a person concept from those slides that they then successfully applied to pictures they had never seen before.

Or had they? D’Amato and van Sant did not stop their analysis simply with the observation that the monkeys had successfully transferred their learning to novel slides – rather they went on to look carefully at the kinds of errors the monkeys had made. Although largely successful with the novel slides, the monkeys made some very puzzling mistakes. For example, one of the person slides that the monkeys had failed to recognize as a picture of a human being had been a head and shoulders portrait – which, to another human, is a classic image of a person. One of the slides that the monkeys had incorrectly classified as containing a human had actually been a shot of a jackal carrying a dead flamingo in its mouth; both the jackal and its prey were also reflected in the water beneath them. What person in her right mind could possible confuse a jackal with a flamingo in its mouth with another human being?

The explanation for both these mistakes is the same: the monkeys had generalized on the basis of the particular features contained in the slides they had been trained with rather than learning the more abstract concept that the experimenters had intended. The head and shoulders portrait of a person lacked the head-torso-arms-legs body shape that had been most common among the images that the monkeys had been trained with, and consequently, they had rejected it as not similar enough to the positive image they were looking for. Similarly, during training, the only slides that had contained flashes of red happened to be those of people. Three of the training slides had contained people wearing a piece of red clothing, whereas none of the nonperson slides had contained the color red. Consequently, when the jackal with prey slide came along during testing, it contained the color red, and so the monkeys classified it as a person slide.

From Wynne & Udell (2013):

Michael Young and colleagues carried out experiments that add to a sense that the pigeon’s perception of pictures of objects is not identical to our own. They trained pigeons to peck in different locations on a computer-controlled touch screen, depending on which of four different objects was presented: an arch, a barrel, a brick, and a triangular wedge (Young et al., 2001). The objects were initially presented to the pigeons as images shaded to suggest light shining on them from one direction. Next, Young and colleagues tested the pigeons with pictures of the same objects, but this time illuminated from a different direction… To the experimenters’ surprise, the pigeons’ ability to recognize the objects was disturbed by changes in lighting that human observers were barely able to perceive… [see below]