Daniel Ellsberg of Pentagon Papers fame recent published a book about his days as a nuclear war planner, The Doomsday Machine. Below are just a few of the bits I found interesting. (There were many others, but they were more difficult to excerpt.)

My first summer [at RAND] I worked seventy-hour weeks, devouring secret studies and analyses till late every night, to get up to speed on the problems and the possible solutions. I was looking for clues as to how we could frustrate the Soviet versions of RAND and SAC, and do it in time to avert a nuclear Pearl Harbor. Or postpone it. From the Air Force intelligence estimates I was newly privy to, and the dark view of the Soviets, which my colleagues shared with the whole national security community, I couldn’t believe that the world would long escape nuclear holocaust. Alain Enthoven and I were the youngest members of the department. Neither of us joined the extremely generous retirement plan RAND offered. Neither of us believed, in our late twenties, we had a chance of collecting on it.

Just one of many stories on how unreliable Ellsberg found command and control procedures to be:

To prevent unauthorized action by a single duty officer with access to Execute codes in any particular command post, there was a universal and supposedly ironclad rule that at least two such officers must be on duty at all times, day and night, and they must both be involved in, and agree on, the authentication of an order to execute nuclear war plans from a higher authority and on their decision to relay this order to subordinate commands… One way or another, each post purported to have arrangements so that one officer by himself could neither authenticate orders received nor send out authenticated Execute commands.

But in practice, not. As various duty officers explained to me, oftentimes only one man was on duty in the office. The personnel requirements for having two qualified officers sitting around in every such station at literally every moment of the night were just too stringent to be met. Duty rosters did provide for it, but not for backups when one officer “had” to be elsewhere—to get some food or for a medical emergency, his own or, on some bases, his wife’s. Did that mean that all subordinate commands would be paralyzed, unable to receive authenticated Execute orders, if the one remaining duty officer received what appeared to be an order to commence nuclear operations during that interval?

That couldn’t be permitted, in the eyes of the officers assigned to this duty, each of whom had faced up to the practical possibility of this situation. So each of them had provided for it “unofficially,” in his own mind or usually by agreement with his fellow duty officers. Each, in reality, had the combinations to both safes, after all, or some arrangement for acquiring them. If there was only one safe, each officer would, in reality, know the full combination to it. One officer would hold both envelopes when the other had to be away. Where there were more elaborate safeguards, the officers had always spent some of their idle hours late at night figuring out how to circumvent them, “if necessary.” They had always succeeded in doing so. I found this in every post I visited.

[Read more…]

From Better Angels of Our Nature, ch. 5:

In “Perpetual Peace,” Kant envisioned a “federation of free states” that would fall well short of an international Leviathan. It would be a gradually expanding club of liberal republics rather than a global megagovernment, and it would rely on the soft power of moral legitimacy rather than on a monopoly on the use of force. The modern equivalent is the intergovernmental organization or IGO — a bureaucracy with a limited mandate to coordinate the policies of participating nations in some area in which they have a common interest. The international entity with the best track record for implementing world peace is probably not the United Nations, but the European Coal and Steel Community, an IGO founded in 1950 by France, West Germany, Belgium, the Netherlands, and Italy to oversee a common market and regulate the production of the two most important strategic commodities. The organization was specifically designed as a mechanism for submerging historic rivalries and ambitions — especially West Germany’s — in a shared commercial enterprise. The Coal and Steel Community set the stage for the European Economic Community, which in turn begot the European Union.

Many historians believe that these organizations helped keep war out of the collective consciousness of Western Europe. By making national borders porous to people, money, goods, and ideas, they weakened the temptation of nations to fall into militant rivalries, just as the existence of the United States weakens any temptation of, say, Minnesota and Wisconsin to fall into a militant rivalry. By throwing nations into a club whose leaders had to socialize and work together, they enforced certain norms of cooperation. By serving as an impartial judge, they could mediate disputes among member nations. And by holding out the carrot of a vast market, they could entice applicants to give up their empires (in the case of Portugal) or to commit themselves to liberal democracy (in the case of former Soviet satellites and, perhaps soon, Turkey).

Richard Clarke and R.P. Eddy recently published Warnings, a book in which they try to identify “those rare people who… have accurate visions of looming disasters.” The opening chapter explains the aims of the book:

…this book will seek to answer these questions: How can we detect a real Cassandra among the myriad of pundits? What methods, if any, can be employed to better identify and listen to these prophetic warnings? Is there perhaps a way to distill the direst predictions from the surrounding noise and focus our attention on them?

…As we proceeded through these Cassandra Event case studies in a variety of different fields, we began to notice common threads: characteristics of the Cassandras, of their audiences, and of the issues that, when applied to a modern controversial prediction of disaster, might suggest that we are seeing someone warning of a future Cassandra Event. By identifying those common elements and synthesizing them into a methodology, we create what we call our Cassandra Coefficient, a score that suggests to us the likelihood that an individual is indeed a Cassandra whose warning is likely accurate, but is at risk of being ignored.

Having established this process for developing a Cassandra Coefficient based on past Cassandra Events, we next listen for today’s Cassandras. Who now among us may be accurately warning us of something we are ignoring, perhaps at our own peril?

Of the risks covered in the book, Clarke says he’s most worried about sea level rise, and Eddy says he’s most worried about superintelligence.

Below is a sampling of what they say in the chapter on risks from advanced AI systems. Note that I’m merely quoting from their take, not necessarily agreeing with it. (Indeed, there are significant parts I disagree with.)

[Read more…]

From What Happened, p. 241:

Technologists like Elon Musk, Sam Altman, and Bill Gates, and physicists like Stephen Hawking have warned that artificial intelligence could one day pose an existential security threat. Musk has called it “the greatest risk we face as a civilization.” Think about it: Have you ever seen a movie where the machines start thinking for themselves that ends well? Every time I went out to Silicon Valley during the campaign, I came home more alarmed about this. My staff lived in fear that I’d start talking about “the rise of the robots” in some Iowa town hall. Maybe I should have. In any case, policy makers need to keep up with technology as it races ahead, instead of always playing catch-up.

Update 11/24/2017: Clinton said more about AI fears in an interview with Hugh Hewitt:

Bill Gates, Elon Musk, Stephen Hawking, a lot of really smart people are sounding an alarm that we’re not hearing. And their alarm is artificial intelligence is not our friend. It can assist us in many ways if it is properly understood and contained. But we are racing headfirst into a new era of artificial intelligence that is going to have dramatic effects on how we live, how we think, how we relate to each other. You know, what are we going to do when we get driverless cars? It sounds like a great idea. And how many millions of people, truck drivers and parcel delivery people and cab drivers and even Uber drivers, what do we do with the millions of people who will no longer have a job? We are totally unprepared for that. What do we do when we are connected to the internet of things and everything we know and everything we say and everything we write is, you know, recorded somewhere? And it can be manipulated against us. So I, you know, one thing I wanted to do if I had been president was to have a kind of blue ribbon commission with people from all kinds of expertise coming together to say what should America’s policy on artificial intelligence be?

But of course, the worries Gates & Musk & Hawking have expressed are not about self-driving cars.