Nano (1996), by Ed Regis, tells the story of nanotechnology up to 1995, and serves as a cautionary tale for others trying to promote the development of a novel science from outside the establishment. Recommended.
The Visioneers (2012), by W. Patrick McCray, tells the story of both Gerard O’Neill — who advocated space colonization in the 1970s — and Eric Drexler, pioneer of nanotechnology. It’s less detailed than Nano, but also recommended.
Soldiers of Reason (2009), by Alex Abella, is a history of RAND Corporation. Most of it is pretty interesting, especially if you happen to run a nonprofit research institute, though I disagree with the author about the purpose and value of rational choice theory.
David and Goliath (2013), by Malcolm Gladwell, includes some great stories as usual but is also Gladwell’s most annoying, disingenuous book yet.
Think Like a Freak (2014), by Levitt & Dubner, also includes some great stories, and is less annoying than David and Goliath, but is basically a repackaging of posts from their blog.
A Troublesome Inheritance (2014), by Nicholas Wade, is about race, genes, and IQ, a touchy subject!
The Knowledge (2014), by Lewis Dartnell, is about what we’d need to know, and what knowledge we’d need to preserve, to reboot human civilization as quickly as possible after some kind of apocalypse. I wish there was more available on this subject. Recommended.
The Up Side of Down (2014), by Megan McArdle, is about how failing well is the key to success. I’ll echo Robin Hanson: “Overall I found it very hard to disagree with anything that McArdle said in her book. If you know me, that is quite some praise.”
From Gladwell’s David and Goliath:
A regulation basketball court is ninety-four feet long. Most of the time, a team would defend only about twenty-four feet of that, conceding the other seventy feet. Occasionally teams played a full-court press—that is, they contested their opponent’s attempt to advance the ball up the court. But they did it for only a few minutes at a time. It was as if there were a kind of conspiracy in the basketball world about the way the game ought to be played, Ranadivé thought, and that conspiracy had the effect of widening the gap between good teams and weak teams. Good teams, after all, had players who were tall and could dribble and shoot well; they could crisply execute their carefully prepared plays in their opponent’s end. Why, then, did weak teams play in a way that made it easy for good teams to do the very things that they were so good at?
Ranadivé looked at his girls. Morgan and Julia were serious basketball players. But Nicky, Angela, Dani, Holly, Annika, and his own daughter, Anjali, had never played the game before. They weren’t all that tall. They couldn’t shoot. They weren’t particularly adept at dribbling. They were not the sort who played pickup games at the playground every evening. Ranadivé lives in Menlo Park, in the heart of California’s Silicon Valley. His team was made up of, as Ranadivé put it, “little blond girls.” These were the daughters of nerds and computer programmers. They worked on science projects and read long and complicated books and dreamed about growing up to be marine biologists. Ranadivé knew that if they played the conventional way—if they let their opponents dribble the ball up the court without opposition—they would almost certainly lose to the girls for whom basketball was a passion. Ranadivé had come to America as a seventeen-year-old with fifty dollars in his pocket. He was not one to accept losing easily. His second principle, then, was that his team would play a real full-court press—every game, all the time. The team ended up at the national championships. “It was really random,” Anjali Ranadivé said. “I mean, my father had never played basketball before.”
From Brafman’s Sway:
The most flattering way to describe the Gator [football] team upon Spurrier’s arrival in 1990 was as a “fixer-upper.” The team had never won a conference title; in fact, it was on probation because of allegations of rule violations by the team’s former coach.
… Spurrier’s most important move was to identify a weak spot in the strategy employed by his opponents. For year the teams in the conference had adhered to a “war of attrition” game strategy: they called conservative plays and held on to the ball for as long as they could, hoping to win a defensive battle…
… Spurrier came to dominate the conference by… introducing what he called the ‘Fun-n-Gun” approach…
Spurrier mixed things up with a generous helping of “big chance plays, where you got to give your players a shot.” In other words, Spurrier’s team passed more often, played more aggressively, and tried to score more touchdowns.
..Spurrier gained an advantage because the other coaches were focused on trying to avoid a potential loss. Think of what it’s like to be a college football coach. As you walk around town, passing fans offer themselves up as instant experts on the game — never afraid to give you a piece of their minds on what you did wrong in yesterday’s match-up. You make one bad move and you get skewered by fans and commentators alike. Meanwhile, ticcket sale revenues, your school’s alumni fundraising, and your job all depend heavily on the football team’s success. All of that pressure adds up… the losses loom large…
You’d have thought that after losing a few games to a team like [the Gators]… the [other] coaches would have reevaluated their war-of-attrition model. But they didn’t. And so Spurrier and his Gators continued to dominate former powerhouses like Alabama, Tennessee, and Auburn. Over the next six years, the coach and his team went on to win four division titles, culminating in the national championship.
- MIRI’s first technical report on the value-loading problem in FAI in a long time.
- Wilson Center & MIT, “Creating a Research Agenda for the Ecological Implications of Synthetic Biology.”
- Wars & Metternich argue in Foreign Policy that “predicting the future is easier than it looks.”
- GiveWell lays out their reasoning on why some U.S. policy areas look more promising for philanthropic intervention than others.
- More evidence that disagreement is less resolvable when people are thinking about their positions in far mode.
- On the unusual effectiveness of logic in computer science.
- Mandiant’s 2014 cybsecurity trends report.
“The stabilization of environments” is a paper about AIs that reshape their environments to make it easier to achieve their goals. This is typically called enforcement, but they prefer the term stabilization because it “sounds less hostile.”
“I’ll open the pod bay doors, Dave, but then I’m going to stabilize the ship… ”
I suppose that’s more likely to provoke public discussion, but… will much good will come of that public discussion? The public had a needless freak-out about in vitro fertilization back in the 60s and 70s and then, as soon as the first IVF baby was born in 1978, decided they were in favor of it.
Someone recently suggested I use an “onion strategy” for the discussion of novel technological risks. The outermost layer of the communication onion would be aimed at the general public, and focus on benefits rather than risks, so as not to provoke an unproductive panic. A second layer for a specialist audience could include a more detailed elaboration of the risks. The most complete discussion of risks and mitigation options would be reserved for technical publications that are read only by professionals.
Eric Drexler seems to wish he had more successfully used an onion strategy when writing about nanotechnology. Engines of Creation included frank discussions of both the benefits and risks of nanotechnology, including the “grey goo” scenario that was discussed widely in the media and used as the premise for the bestselling novel Prey.
Ray Kurzweil may be using an onion strategy, or at least keeping his writing in the outermost layer. If you look carefully, chapter 8 of The Singularity is Near takes technological risks pretty seriously, and yet it’s written in such a way that most people who read the book seem to come away with an overwhelmingly optimistic perspective on technological change.
George Church may be following an onion strategy. Regenesis also contains a chapter on the risks of advanced bioengineering, but it’s presented as an “epilogue” that many readers will skip.
Perhaps those of us writing about AGI for the general public should try to discuss:
- astronomical stakes rather than existential risk
- Friendly AI rather than AGI risk or the superintelligence control problem
- the orthogonality thesis and convergent instrumental values and complexity of values rather than “doom by default”
MIRI doesn’t have any official recommendations on the matter, but these days I find myself leaning toward an onion strategy.
Immediately before launching this new site I was posting regular assorted links to Facebook. I’ve collected those links below.
May 31st links
- Scott Aaronson’s reply to Giulio Tononi’s reply to Scott Aaronson on the integrated information theory of consciousness.
- Equity market guru accuracy ratings, based on 6500+ forecasts from 68 gurus.
- Machines vs. lawyers.
- Robin Goldstein pranks Wine Spectator with a fake restaurant.
- Steven Pinker’s next book is available for pre-order.
May 29th links
- FLI’s inaugural talks and panel: “The future of technology, benefits and risks” (video).
- How much do Y Combinator founders earn?
- Megaprojects (> $1B budget) almost never finish on time, on budget, and with the promised benefits.
- Google has a working prototype of a fully self-driving car with no steering wheel (video).
- SIGGRAPH 2014 technical papers preview (video).
May 27th links
- Sandberg, “Smarter policymaking through improved collective cognition.”
- Johann Schumann on high-assurance systems.
- “Communicating values to autonomous agents.”
- Robotic arm catches everything tossed in its direction (video).
Nano (p. 113) quotes Eric Drexler describing the time he first met Richard Feynman at a party:
We talked about the PNAS article [on nanotechnology], and generally he indicated that, yeah, this was a sensible thing… at one point I was talking about the need for institutions to handle some of the problems [nanotechnology] raised, and [Feynman] remarked [that] institutions were made up of people and therefore of fools.
Feynman sounds downright Yudkowskian on this point, if you ask me.