Over the years, my colleagues and I have spoken to many machine learning researchers who, perhaps after some discussion and argument, claim to think there’s a moderate chance — a 5%, or 15%, or even a 40% chance — that AI systems will destroy human civilization in the next few decades. However, I often detect what Bryan Caplan has called a “missing mood“; a mood they would predictably exhibit if they really thought such a dire future was plausible, but which they don’t seem to exhibit. In many cases, the researcher who claims to think that medium-term existential catastrophe from AI is plausible doesn’t seem too upset or worried or sad about it, and doesn’t seem to be taking any specific actions as a result.

Not so with Elon Musk. Consider his reaction (here and here) when podcaster Joe Rogan asks about his AI doomsaying. Musk stares at the table, and takes a deep breath. He looks sad. Dejected. Fatalistic. Then he says:

I tried to convince people to slow down AI, to regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened. Nobody listened… Maybe [one day] they will [listen]. So far they haven’t.

…Normally the way regulations work is very slow… Usually it’ll be something, some new technology, it will cause damage or death, there will be an outcry, there will be an investigation, years will pass, there will be some kind of insight committee, there will be rulemaking, then there will be oversight, eventually regulations. This all takes many years… This timeframe is not relevant to AI. You can’t take 10 years from the point at which it’s dangerous. It’s too late.

……I was warning everyone I could. I met with Obama, for just one reason [to talk about AI danger]. I met with Congress. I was at a meeting of all 50 governors, I talked about AI danger. I talked to everyone I could. No one seemed to realize where this was going.

Moreover, I believe Musk when he says that his ultimate purpose for founding Neuralink is to avert an AI catastrophe: “If you can’t beat it, join it.” Personally, I’m not optimistic that brain-computer interfaces can avert AI catastrophe — for roughly the reasons outlined in the BCIs section of Superintelligence ch. 2 — but Musk came to a different assessment, and I’m glad he’s trying.

Whatever my disagreements with Musk (I have plenty), it looks to me like Musk doesn’t just profess concern about AI existential risk. I think he feels it in his bones, when he wakes up in the morning, and he’s spending a significant fraction of his time and capital to try to do something about it. And for that I am grateful.

Most people around the world — except for residents of a handful of competent countries such as New Zealand, Vietnam, and Rwanda — have now spent an entire year watching their government fail miserably to prepare for and respond to a very predictable (and predicted) pandemic, for example by:

• sending masks to everyone, and promising to buy lots of masks
• testing tons of people regularly and analyzing the results
• doing contract tracing
• promising to buy tons of vaccine doses, very early
• setting up vaccination facilities, and making them trivially easy to find and use

My friend and colleague Daniel Dewey recently noted that it seems like private actors could have greatly mitigated the impact of the pandemic by creating in advance a variety of “state capacity backups,” i.e. organizations that are ready to do the things we’d want governments to do, if a catastrophe strikes and government response is ineffective.

A state capacity backup could do some things unilaterally (e.g. stockpile and ship masks), and in other cases it could offer its services to governments for functions it can’t perform without state sign-off (e.g. setting up vaccination facilities).

I would like to see more exploration of this idea, including analyses of past examples of privately-provided “state capacity backups” and how well they worked.

Let’s say you want to know how likely it is that an innovative new product will succeed, or that China will invade Taiwan in the next decade, or that a global pandemic will sweep the world — basically any question for which you can’t just use “predictive analytics,” because you don’t have a giant dataset you can plug into some statistical models like (say) Amazon can when predicting when your package will arrive.

Is it possible to produce reliable, accurate forecasts for such questions?

Somewhat amazingly, the answer appears to be “yes, if you do it right.”

Prediction markets are one promising method for doing this, but they’re mostly illegal in the US, and various implementation problems hinder their accuracy for now. Fortunately, there is also the “superforecasting” method, which is completely legal and very effective.

How does it work? The basic idea is very simple. The steps are:

1. First, bother to measure forecasting accuracy at all. Some industries care a lot about their forecasting accuracy and therefore measure it, for example hedge funds. But most forecasting-heavy industries don’t make much attempt to measure their forecasting accuracy, for example journalism, philanthropy, scientific research, or the US intelligence community.
2. Second, identify the people who are consistently more accurate than everyone else — say, those in the top 0.1% for accuracy, for multiple years in a row (without regression to the mean). These are your “superforecasters.”
3. Finally, pose your forecasting questions to the superforecasters, and use an aggregate of their predictions.

Technically, the usual method is a bit more complicated than that, but these three simple steps are the core of the superforecasting method.

So, how well does this work?

A few years ago, the US intelligence community tested this method in a massive, rigorous forecasting tournament that included multiple randomized controlled trials and produced over a million forecasts on >500 geopolitical forecasting questions such as “Will there be a violent incident in the South China Sea in 2013 that kills at least one person?” This study found that:

1. This method produced forecasts that were very well-calibrated, in the sense that forecasts made with 20% confidence came true 20% of the time, forecasts made with 80% confidence came true 80% of the time, and so on. The method is not a crystal ball; it can’t tell you for sure whether China will invade Taiwan in the next decade, but if it tells you there’s a 10% chance, then you can be pretty confident the odds really are pretty close to 10%, and decide what policy is appropriate given that level of risk.
2. This method produced forecasts that were far more accurate than those of a typical forecaster or other approaches that were tried.

Those are pretty amazing results! And from an unusually careful and rigorous study, no less!

So you might think the US intelligence community has eagerly adopted the superforecasting method, especially since the study was funded by the intelligence community, specifically for the purpose of discovering ways to improve the accuracy of US intelligence estimates used by policymakers to make tough decisions. Unfortunately, in my experience, very few people in the US intelligence and national security communities have even heard of these results, or even the term “superforecasting.”

A large organization such as the CIA or the Department of Defense has enough people, and makes enough forecasts, that it could implement all steps of the superforecasting method itself, if it wanted to. Smaller organizations, fortunately, can just contract already-verified superforecasters to make well-calibrated forecasts about the questions of greatest importance to their decision-making. In particular:

• The superforecasters who out-predicted intelligence community analysts in the forecasting tournament described above are available to be contracted through Good Judgment Inc.
• Another company, Hypermind, offers aggregated forecasts from “champion forecasters,” i.e. the most accurate forecasters across thousands of forecasting questions for corporate clients going back (in some cases) almost two decades.
• Several other projects, for example Metaculus, are also beginning to identify forecasters with unusually high accuracy across hundreds of questions.

These companies each have their own strengths and weaknesses, and Open Philanthropy has commissioned forecasts from all three in the past couple years. If you work for a small organization that regularly makes important decisions based on what you expect to happen in the future, including what you expect to happen if you make one decision vs. another, I suggest you try them out. (All three offer “conditional” questions, e.g. “What’s the probability of outcome X if I make decision A, and what’s the probability of that same outcome if I instead make decision B?”)

If you work for an organization that is very large and/or works with highly sensitive information, for example the CIA, you should consider implementing the entire superforecasting process internally. (Though contracting one or more of the above organizations might be a good way to test the model cheaply before going all-in.)

I’ve noticed some practical differences in the challenges and conveniences faced by consumers of rock, jazz, and (“Western”) classical music recordings.

General notes:

• Overall, I think it’s easiest and most convenient to be a consumer of rock recordings, somewhat harder to be a consumer of jazz recordings, and much harder to be a consumer of classical recordings.
• By “classical” I mean to include contemporary classical.
• Pop and hip-hop and (rock-descended) electronic music mostly follow the rock model. I’m less familiar with the markets for recordings of folk musics and “non-Western classical” musics.

Covers

• Rock: Cover songs are fairly rare, especially on studio albums (as compared to live recordings).
• Jazz: Cover tracks are common, including on studio albums. Many of the most popular and/or well-regarded jazz albums consist largely or mostly of covers.
• Classical: Almost all tracks are cover tracks, especially if weighting by sales.

Performers/composers

• Rock: One or more of the performers are typically also the composers, though this is less true at the big-label pop end of the spectrum.
• Jazz: Except for the covers, one of the performers is usually the composer, though the composer plays a smaller role than in rock because improvisation is a major aspect.
• Classical: Composers rarely perform their own work on recordings.

Labeling/attribution

• Rock: Simple artist + album/track labeling, because the composer(s) and performer(s) are often entirely or partly the same, and tracks composed by a single member of the band are just attributed to the band (e.g. “The Beatles” instead of “John Lennon” or “Paul McCartney”).
• Jazz: Albums and tracks are usually labeled according to the performer, even when the composer is someone else. (E.g. Closer is attributed to Paul Bley even though Carla Bley composed most of it.)
• Classical: Album titles might be a list of all pieces on the album, or the title of just one of several pieces on the album, or something else. As for the “artist,” on the cover art and/or in online stores/services/databases, sometimes the composer(s) will be emphasized, sometimes the performer(s) will be emphasized, and sometimes the conductor will be emphasized. For any given album, it might be listed under the composer in one store/service/database, listed under the composer in another store/service/database, and listed under the performer(s) under a third store/service/database.

Canonical recordings

• Rock: Most pieces (identified by artist+song) have one canonical recording, usually the version from first studio album it appeared on. So when people refer to a piece by artist+song, everyone is talking about the same thing.
• Jazz: Many pieces (identified by performer+piece or composer+piece) lack a canonical recording, because different versions of it often appear on multiple recordings by the same performer, sometimes the earliest version is not the most popular version, and consumers and critics disagree on which version is best.
• Classical: For the most part, only less-popular contemporary pieces (identified by composer+piece) have a canonical recording. Everything else typically lacks a canonical recording because the earliest recording is rarely the most popular version, and consumers and critics disagree on which recording of a piece is best.

Genre tags

• Rock: Hundreds of narrow and informative genre tags are in wide use, e.g. not just “metal” but “death metal” and even “technical death metal.”
• Jazz: Only a couple dozen genre tags are in wide use, so it can be very hard to know from genre tags what an album will sound like. Different albums labeled simply “avant-garde jazz” or “post-bop” or “jazz fusion” can sound extremely different from each other.
• Classical: Only a couple dozen genre tags are in wide use, so it can be very hard to know from genre tags what a piece will sound like. Moreover, classical music after ~1910 is far more unique on average (per piece) than rock or jazz, because the incentives for innovation are higher, so classical music after ~1910 “needs” more genre tags than rock or jazz.

Ratings

• Rock: Reviewers often provide a quick-take rating, e.g. “3 out of 5 stars” or “8.5/10,” which makes it easier for you to filter for music you might like.
• Jazz: Quick-take ratings from reviewers are uncommon but not rare.
• Classical: Quick-take ratings from reviewers are fairly rare.

Availability

• Rock: Most tracks are recorded and released within a few years of being composed.
• Jazz: Most tracks are recorded and released within a few years of being composed.
• Classical: Even after the invention of cheap recording equipment and cheap release methods, very few pieces are recorded and released within 5 years of being composed.

(I’ve now re-organized this post by feature rather than by super-genre.)