Wish I could get my hands on this:

Doug knows that when people read for pleasure they naturally gravitate to the like-minded. So he created a database containing hundreds of information sources—from the New York Times to obscure blogs—that are tagged by their ideological orientation, subject matter, and geographical origin, then wrote a program that selects what he should read next using criteria that emphasize diversity. Thanks to Doug’s simple invention, he is sure to constantly encounter different perspectives.

A case study in changed decision-making methods:

If the Bay of Pigs was the Kennedy administration’s nadir, the Cuban missile crisis was its zenith, a moment when Kennedy and his team creatively engineered a positive result under extreme pressure. Knowing this, we might assume Kennedy cleaned house after the Bay of Pigs and surrounded himself with far superior advisers in time for the missile crisis. But he didn’t. The cast of characters in both dramas is mostly the same: the team that bungled the Bay of Pigs was the team that performed brilliantly during the Cuban missile crisis.

…After the fiasco [of the Bay of Pigs], Kennedy ordered an inquiry to figure out how his people could have botched it so badly. It identified cozy unanimity as the key problem and recommended changes to the decision-making process to ensure it could never develop again. Skepticism was the new watchword. Participants were to speak not only as specialists in their area of expertise but as generalists, with a license to question anything. Special counsel Theodore Sorensen and the president’s brother Bobby were designated “intellectual watchdogs,” whose job was to “pursue relentlessly every bone of contention in order to prevent errors arising from too superficial an analysis of the issues,” Janis [author of the original book on groupthink] noted. “Accepting this role avidly, Robert Kennedy, at the expense of becoming unpopular with some of his associates, barked out sharp and sometimes rude questions. Often, he deliberately became the devil’s advocate.” Protocol and hierarchy would impede these freewheeling discussions, so they were set aside. New advisers were occasionally brought in to provide fresh perspectives. And John F. Kennedy would sometimes leave the room to let the group talk things through, knowing that there was less true give-and-take when the president was present. That last consideration was crucial. Kennedy started the crisis thinking that, at a minimum, he had to authorize preemptive air attacks on the Soviet missile launchers, but he kept that to himself so it wouldn’t be the focus of the discussion. As a result, “by the end of the first day of meetings the committee had seriously discussed ten alternatives,” and the president’s thinking started to change. It was never easy. There were constant disagreements. The stress was brutal. But it was a process that led to a negotiated peace, not nuclear war.

An interesting memo:

On April 11, 2001, Defense Secretary Donald Rumsfeld sent a memo to President George W. Bush and Vice President Dick Cheney. “I ran across this piece on the difficulty of predicting the future,” Rumsfeld wrote. “I thought you might find it interesting.” The “piece” looks at the strategic situation at the start of each decade between 1900 and 2000 and shows that, in every case, the reality was a stunning change from ten years earlier.

Here is the text of the memo, written by Lin Wells:

Thoughts for the 2001 Quadrennial Defense Review

• If you had been a security policy-maker in the world’s greatest power in 1900, you would have been a Brit, looking warily at your age-old enemy, France.
• By 1910, you would be allied with France and your enemy would be Germany.
• By 1920, World War I would have been fought and won, and you’d be engaged in a naval arms race with your erstwhile allied, the U.S. and Japan.
• By 1930, naval arms limitation treaties were in effect, the Great Depression was underway, and the defense planning standard said “no war for ten years.”
• Nine years later World War II had begun.
• By 1950, Britain no longer was the world’s greatest power, the Atomic Age had dawned, and a “police action” was underway in Korea.
• Ten years later the political focus was on the “missile gap,”” the strategic paradigm was shifting from massive retaliation to flexible response, and few people had heard of Vietnam.
• By 1970, the peak of our involvement in Vietnam had come and gone, we were beginning détente with the Soviets, and we were anointing the Shah as our protégé in the Gulf region.
• By 1980, the Soviets were in Afghanistan, Iran was in the throes of revolution, there was talk of our “hollow forces” and a “window of vulnerability,” and the U.S. was the greatest creditor nation the world had ever seen.
• By 1990, the Soviet Union was within a year of dissolution, American forces in the Desert were on the verge of showing they were anything but hollow, teh U.S. had become the greatest debtor nation the world had ever known, and almost no one had heard of the internet.
• Ten years later, Warsaw was the capital of a NATO nation, asymmetric threats transcended geography, and the parallel revolutions of information, biotechnology, robotics, nanotechnology, and high density energy sources foreshadowed changes almost beyond forecasting.
• All of which is to say that I’m not sure what 2010 will look like, but I’m sure that it will be very little like we expect, so we should plan accordingly.

And finally, Tetlock on answering big questions by answering lots of little questions:

In the spring of 2013 I met with Paul Saffo, a Silicon Valley futurist and scenario consultant. Another unnerving crisis was brewing on the Korean peninsula, so when I sketched the forecasting tournament for Saffo, I mentioned a question IARPA had asked: Will North Korea “attempt to launch a multistage rocket between 7 January 2013 and 1 September 2013?” Saffo thought it was trivial. A few colonels in the Pentagon might be interested, he said, but it’s not the question most people would ask. “The more fundamental question is ‘How does this all turn out?’ ” he said. “That’s a much more challenging question.”

So we confront a dilemma. What matters is the big question, but the big question can’t be scored. The little question doesn’t matter but it can be scored, so the IARPA tournament went with it. You could say we were so hell-bent on looking scientific that we counted what doesn’t count.

That is unfair. The questions in the tournament had been screened by experts to be both difficult and relevant to active problems on the desks of intelligence analysts. But it is fair to say these questions are more narrowly focused than the big questions we would all love to answer, like “How does this all turn out?” Do we really have to choose between posing big and important questions that can’t be scored or small and less important questions that can be? That’s unsatisfying. But there is a way out of the box.

Implicit within Paul Saffo’s “How does this all turn out?” question were the recent events that had worsened the conflict on the Korean peninsula. North Korea launched a rocket, in violation of a UN Security Council resolution. It conducted a new nuclear test. It renounced the 1953 armistice with South Korea. It launched a cyber attack on South Korea, severed the hotline between the two governments, and threatened a nuclear attack on the United States. Seen that way, it’s obvious that the big question is composed of many small questions. One is “Will North Korea test a rocket?” If it does, it will escalate the conflict a little. If it doesn’t, it could cool things down a little. That one tiny question doesn’t nail down the big question, but it does contribute a little insight. And if we ask many tiny-but-pertinent questions, we can close in on an answer for the big question. Will North Korea conduct another nuclear test? Will it rebuff diplomatic talks on its nuclear program? Will it fire artillery at South Korea? Will a North Korean ship fire on a South Korean ship? The answers are cumulative. The more yeses, the likelier the answer to the big question is “This is going to end badly.”

I call this Bayesian question clustering because of its family resemblance to the Bayesian updating discussed in chapter 7. Another way to think of it is to imagine a painter using the technique called pointillism. It consists of dabbing tiny dots on the canvas, nothing more. Each dot alone adds little. But as the dots collect, patterns emerge. With enough dots, an artist can produce anything from a vivid portrait to a sweeping landscape.

There were question clusters in the IARPA tournament, but they arose more as a consequence of events than a diagnostic strategy. In future research, I want to develop the concept and see how effectively we can answer unscorable “big questions” with clusters of little ones.