Treacherous turns in the wild

Bostrom (2014) worries about an AI “treacherous turn”:

…one idea for how to ensure superintelligence safety… is that we validate the safety of a superintelligent AI empirically by observing its behavior while it is in a controlled, limited environment (a “sandbox”) and that we only let the AI out of the box if we see it behaving in a friendly, cooperative, responsible manner. The flaw in this idea is that behaving nicely while in the box is a convergent instrumental goal for friendly and unfriendly AIs alike. An unfriendly AI of sufficient intelligence realizes that its unfriendly final goals will be best realized if it behaves in a friendly manner initially, so that it will be let out of the box. It will only start behaving in a way that reveals its unfriendly nature when it no longer matters whether we find out; that is, when the AI is strong enough that human opposition is ineffectual.

Some people have told me they think this is unrealistic, apparently even for a machine superintelligence far more capable than any current AI system. But I learned today that this has already happened at least once before —  twenty years ago!

As part of a project studying the evolution of (simulated) organisms, computer scientist Charles Ofria wanted to limit the replication rate of a digital organism. So, he programmed the system to pause after each mutation, measure the mutant’s replication rate in an isolated test environment, and delete the mutant if it replicated faster than its parent. However, the organisms evolved to recognize when they were in the test environment and “play dead” (pause replication) so they would not be eliminated and instead be kept in the population where they could continue to replicate outside the test environment. Once he discovered this, Ofria randomized the inputs of the test environment so that it couldn’t be so easily detected, but the organisms evolved a new strategy, to probabilistically perform tasks that would accelerate their replication, thus slipping through the test environment some percentage of the time and continuing to accelerate their replication thereafter.

[Read more…]

Media diet for Q1 2021

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Music I most enjoyed discovering this quarter:

[Read more…]

Media I’m looking forward to, Q2 2021 edition

Added this quarter:

  • Johnson, Extra Life (May 2021)
  • Grossman, How Social Science Got Better (Aug 2021)
  • Buchanan & Imbrie, The New Fire (Mar 2022)
  • Kemp, Downfall (TBD)
  • Rick and Morty: season 5 (Jun 2021)
  • Better Call Saul: season 6 (TBD 2022)
  • Curb Your Enthusiasm: season 11 (TBD)
  • Succession: season 3 (TBD 2021)
  • Barry: season 3 (TBD)
  • Atlanta: season 3 (TBD)
  • Master of None: season 3 (TBD)

Books

bold = especially excited

[Read more…]

State capacity backups

Most people around the world — except for residents of a handful of competent countries such as New Zealand, Vietnam, and Rwanda — have now spent an entire year watching their government fail miserably to prepare for and respond to a very predictable (and predicted) pandemic, for example by:

  • sending masks to everyone, and promising to buy lots of masks
  • testing tons of people regularly and analyzing the results
  • doing contract tracing
  • promising to buy tons of vaccine doses, very early
  • setting up vaccination facilities, and making them trivially easy to find and use

My friend and colleague Daniel Dewey recently noted that it seems like private actors could have greatly mitigated the impact of the pandemic by creating in advance a variety of “state capacity backups,” i.e. organizations that are ready to do the things we’d want governments to do, if a catastrophe strikes and government response is ineffective.

A state capacity backup could do some things unilaterally (e.g. stockpile and ship masks), and in other cases it could offer its services to governments for functions it can’t perform without state sign-off (e.g. setting up vaccination facilities).

I would like to see more exploration of this idea, including analyses of past examples of privately-provided “state capacity backups” and how well they worked.

Superforecasting in a nutshell

Let’s say you want to know how likely it is that an innovative new product will succeed, or that China will invade Taiwan in the next decade, or that a global pandemic will sweep the world — basically any question for which you can’t just use “predictive analytics,” because you don’t have a giant dataset you can plug into some statistical models like (say) Amazon can when predicting when your package will arrive.

Is it possible to produce reliable, accurate forecasts for such questions?

Somewhat amazingly, the answer appears to be “yes, if you do it right.”

Prediction markets are one promising method for doing this, but they’re mostly illegal in the US, and various implementation problems hinder their accuracy for now. Fortunately, there is also the “superforecasting” method, which is completely legal and very effective.

How does it work? The basic idea is very simple. The steps are:

  1. First, bother to measure forecasting accuracy at all. Some industries care a lot about their forecasting accuracy and therefore measure it, for example hedge funds. But most forecasting-heavy industries don’t make much attempt to measure their forecasting accuracy, for example journalism, philanthropy, or the US intelligence community.
  2. Second, identify the people who are consistently more accurate than everyone else — say, those in the top 0.1% for accuracy, for multiple years in a row. These are your “superforecasters.”
  3. Finally, pose your forecasting questions to the superforecasters, and use an aggregate of their predictions.

Technically, the usual method is a bit more complicated than that, but these three simple steps are the core of the superforecasting method.

So, how well does this work?

A few years ago, the US intelligence community tested this method in a massive, rigorous forecasting tournament that included multiple randomized controlled trials and produced over a million forecasts on >500 geopolitical forecasting questions such as “Will there be a violent incident in the South China Sea in 2013 that kills at least one person?” This study found that:

  1. This method produced forecasts that were very well-calibrated, in the sense that forecasts made with 20% confidence came true 20% of the time, forecasts made with 80% confidence came true 80% of the time, and so on. The method is not a crystal ball; it can’t tell you for sure whether China will invade Taiwan in the next decade, but if it tells you there’s a 10% chance, then you can be pretty confident the odds really are pretty close to 10%, and decide what policy is appropriate given that level of risk.
  2. This method produced forecasts that were far more accurate than those of a typical forecaster or other approaches that were tried, and ~30% more accurate than intelligence community analysts who (unlike the superforecasters ) had access to expensively-collected classified information and years of training in the geopolitical issues they were making forecasts about.

Those are pretty amazing results! And from an unusually careful and rigorous study, no less!

So you might think the US intelligence community has eagerly adopted the superforecasting method, especially since the study was funded by the intelligence community, specifically for the purpose of discovering ways to improve the accuracy of US intelligence estimates used by policymakers to make tough decisions. Unfortunately, in my experience, very few people in the US intelligence and national security communities have even heard of these results, or even the term “superforecasting.”

A large organization such as the CIA or the Department of Defense has enough people, and makes enough forecasts, that it could implement all steps of the superforecasting method itself, if it wanted to. Smaller organizations, fortunately, can just contract already-verified superforecasters to make well-calibrated forecasts about the questions of greatest importance to their decision-making. In particular:

  • The superforecasters who out-predicted intelligence community analysts in the forecasting tournament described above are available to be contracted through Good Judgment Inc.
  • Another company, Hypermind, offers aggregated forecasts from “champion forecasters,” i.e. the most accurate forecasters across thousands of forecasting questions for corporate clients going back (in some cases) almost two decades.
  • Several other projects, for example Metaculus, are also beginning to identify forecasters with unusually high accuracy across hundreds of questions.

These companies each have their own strengths and weaknesses, and Open Philanthropy has commissioned forecasts from all three in the past couple years. If you work for a small organization that regularly makes important decisions based on what you expect to happen in the future, including what you expect to happen if you make one decision vs. another, I suggest you try them out. (All three offer “conditional” questions, e.g. “What’s the probability of outcome X if I make decision A, and what’s the probability of that same outcome if I instead make decision B?”)

If you work for an organization that is very large and/or works with highly sensitive information, for example the CIA, you should consider implementing the entire superforecasting process internally. (Though contracting one or more of the above organizations might be a good way to test the model cheaply before going all-in.)

Media diet for Q4 2020

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Okay, music I most enjoyed discovering this quarter:

[Read more…]

Different challenges faced by consumers of rock, jazz, or classical music recordings

I’ve noticed some practical differences in the challenges and conveniences faced by consumers of rock, jazz, and (“Western”) classical music recordings.

General notes:

  • Overall, I think it’s easiest and most convenient to be a consumer of rock recordings, somewhat harder to be a consumer of jazz recordings, and much harder to be a consumer of classical recordings.
  • By “classical” I mean to include contemporary classical.
  • Pop and hip-hop and (rock-descended) electronic music mostly follow the rock model. I’m less familiar with the markets for recordings of folk musics and “non-Western classical” musics.

Covers

  • Rock: Cover songs are fairly rare, especially on studio albums (as compared to live recordings).
  • Jazz: Cover tracks are common, including on studio albums. Many of the most popular and/or well-regarded jazz albums consist largely or mostly of covers.
  • Classical: Almost all tracks are cover tracks, especially if weighting by sales.

Performers/composers

  • Rock: One or more of the performers are typically also the composers, though this is less true at the big-label pop end of the spectrum.
  • Jazz: Except for the covers, one of the performers is usually the composer, though the composer plays a smaller role than in rock because improvisation is a major aspect.
  • Classical: Composers rarely perform their own work on recordings.

Labeling/attribution

  • Rock: Simple artist + album/track labeling, because the composer(s) and performer(s) are often entirely or partly the same, and tracks composed by a single member of the band are just attributed to the band (e.g. “The Beatles” instead of “John Lennon” or “Paul McCartney”).
  • Jazz: Albums and tracks are usually labeled according to the performer, even when the composer is someone else. (E.g. Closer is attributed to Paul Bley even though Carla Bley composed most of it.)
  • Classical: Album titles might be a list of all pieces on the album, or the title of just one of several pieces on the album, or something else. As for the “artist,” on the cover art and/or in online stores/services/databases, sometimes the composer(s) will be emphasized, sometimes the performer(s) will be emphasized, and sometimes the conductor will be emphasized. For any given album, it might be listed under the composer in one store/service/database, listed under the composer in another store/service/database, and listed under the performer(s) under a third store/service/database.

Canonical recordings

  • Rock: Most pieces (identified by artist+song) have one canonical recording, usually the version from first studio album it appeared on. So when people refer to a piece by artist+song, everyone is talking about the same thing.
  • Jazz: Many pieces (identified by performer+piece or composer+piece) lack a canonical recording, because different versions of it often appear on multiple recordings by the same performer, sometimes the earliest version is not the most popular version, and consumers and critics disagree on which version is best.
  • Classical: For the most part, only less-popular contemporary pieces (identified by composer+piece) have a canonical recording. Everything else typically lacks a canonical recording because the earliest recording is rarely the most popular version, and consumers and critics disagree on which recording of a piece is best.

Genre tags

  • Rock: Hundreds of narrow and informative genre tags are in wide use, e.g. not just “metal” but “death metal” and even “technical death metal.”
  • Jazz: Only a couple dozen genre tags are in wide use, so it can be very hard to know from genre tags what an album will sound like. Different albums labeled simply “avant-garde jazz” or “post-bop” or “jazz fusion” can sound extremely different from each other.
  • Classical: Only a couple dozen genre tags are in wide use, so it can be very hard to know from genre tags what a piece will sound like. Moreover, classical music after ~1910 is far more unique on average (per piece) than rock or jazz, because the incentives for innovation are higher, so classical music after ~1910 “needs” more genre tags than rock or jazz.

Ratings

  • Rock: Reviewers often provide a quick-take rating, e.g. “3 out of 5 stars” or “8.5/10,” which makes it easier for you to filter for music you might like.
  • Jazz: Quick-take ratings from reviewers are uncommon but not rare.
  • Classical: Quick-take ratings from reviewers are fairly rare.

Availability

  • Rock: Most tracks are recorded and released within a few years of being composed.
  • Jazz: Most tracks are recorded and released within a few years of being composed.
  • Classical: Even after the invention of cheap recording equipment and cheap release methods, very few pieces are recorded and released within 5 years of being composed.

(I’ve now re-organized this post by feature rather than by super-genre.)

Funny or interesting Scaruffi Quotes (part 7)

Previously: 1234, 5, 6.

On White Flight:

White Flight is a cacophonous collage of disparate musical ideas that don’t even try to coexist and make sense together. They simply pile up, one on top of the other, and be the listener the one to make sense of the Babelic confusion. The first two songs are misleading in their melodic simplicity. “Now” is a demented, heavily-arranged aria that sounds like a collaboration between VanDyke Parks and Syd Barrett. “Pastora Divine” is a pastoral psychedelic singalong that Kevin Ayers could have concocted in the 1970s if backed by the Velvet Underground. By the third one, any pretense of logic begins to fall apart. The somnolent sparse blues “Solarsphere” is ripped apart by a roaring hard-rock riff and drowns in ambient-lysergic madness. “The Condition” and the jazz-electronic mayhem of “Timeshaker” evoke the anarchic psychedelic freak-outs of Red Crayola; while the disjointed chant with wah-wah organ of “Oz Icaro” and the brief exotic dance of “Galactic Seed” evoke the acid-folk eruptions of the Holy Modal Rounders, except that Roelofs employs a different generation of devices: breakbeats, digital noise, sound effects, vocal effects, non-rock instruments to conjure a sense of poetic detachment from anything that music is supposed to be. Roelofs ends the album in the tone that is more pensive and philosophical, and musically more convoluted, of “Deathhands” and “The Secret Sound.” His extreme message is the hyper-syncopated drum’n’bass and free-jazz hemorrage of “Superconductor” that ends with a cryptic whistle in a bed of crickets.

[Read more…]

One Billion Americans

I have a lot of controversial views. For example, I think it’s morally better to help others more rather than helping them less (utilitarianism), that people matter equally regardless of their group membership, location in spacetime, etc. (impartiality), that therefore the most important impacts of my actions are spread throughout the long-run future, where the vast majority of people are (longtermism), and that advances in AI this century will probably have a larger (positive or negative) long-run impact on aggregate welfare than anything else (transformative AI focus). Most people strongly disagree with all those views, and often find them offensive.

But not all my views are controversial. One of my least controversial views is that both the US in particular and humanity in general will probably be better off if the US (despite its many deep flaws) remains the world’s leading power, given the available alternatives for global leadership.

Probably the only way for the US to remain the world’s leading power is for the U.S. to dramatically grow its population, especially its high-skill population. As Vox co-founder Matt Yglesias argues in his new book One Billion Americans:

…the big picture idea of [this] book, that America should try to stay number one, already [commands broad consensus in America]. The question is what follows from that.

For starters, it is beyond dispute that there are fewer American people than there are Chinese or Indian people, as is the fact that China and India are trying to become less poor and seem to be succeeding. Maybe they’ll just stumble and fail, in which case we will stay number one. But it would be unfortunate for hundreds of millions of people to be consigned to poverty forever. It’s not an outcome we have it within our power to guarantee. And even if we could, it would be hideously immoral to pursue it.

By contrast, tripling the nation’s population to match the rising Asian powers is something that is in our power to achieve…

…What the various diplomats and admirals and trade negotiators and Asia hands who think about the China question don’t want to admit is that all the diplomacy and aircraft carriers and shrewd trade tactics in the world aren’t going to make a whit of difference if China is just a much bigger and more important country than we are. The original Thirteen Colonies, by the same token, could have made for a nice, quiet, prosperous agricultural nation — like a giant New Zealand. But no number of smart generals could have helped a country like that intervene decisively in World War II.

A more populous America — filled with more immigrants and more children, with its cities repopulated and its construction industry booming—would not be staring down the barrel of inevitable relative decline. We are richer today than China or India. And while we neither can nor should wish for those countries to stay poor, we can become even richer by becoming larger. And by becoming larger we will also break the dynamic whereby growth in Asia naturally means America’s eclipse as the world’s leading power.

The United States has been the number one power in the world throughout my entire lifetime and throughout the living memory of essentially everyone on the planet today. The notion that this state of affairs is desirable and ought to persist is one of the least controversial things you could say in American politics today.

We should take that uncontroversial premise seriously, adopt the logical inference that to stay on top we’re going to need more people — about a billion people — and then follow that inference to where it leads in terms of immigration, family policy and the welfare state, housing, transportation, and more.

Unfortunately, Yglesias doesn’t actually run the numbers on how different immigration and family planning policies might affect U.S. demographics, how that might in turn affect various measures of national power, and what that implies about the likely relative power of the U.S. and China (and India) in different domains and at different times in the 21st century. That would be a difficult and speculative exercise, but I would love to see it done.

In the meantime, I suspect Yglesias is right about the big picture.

(But, on the details, I roughly agree with some of Caplan’s criticisms, along with some points others have made.)

Media diet for Q3 2020

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Okay, music I most enjoyed discovering this quarter:

[Read more…]

Media I’m looking forward to, Q4 2020 edition

Added this quarter:

Books

bold = especially excited

[Read more…]

Funny or interesting Scaruffi Quotes (part 6)

Previously: 1234, 5.

On Amon Tobin:

Amon Tobin well impersonated the classical composer in the hip-hop age. Instead of composing symphonies for orchestras, Tobin glued together sonic snippets using electronic and digital equipment. Adventures in Foam (1996)… and especially his aesthetic manifesto and masterpiece, Bricolage (1997), unified classical, jazz, rock and dance music in a genre and style that was universal. Tobin warped the distinctive timbres of instruments to produce new kinds of instruments, and then wove them into an organic flow of sound. Tobin kept refining his art of producing amazingly sophisticated and seamless puzzles on Permutation (1998), Supermodified (2000) and, best of his second phase, Out From Out Where (2002). Once he had exhausted the possibilities of instruments and samples, Tobin turned to found sounds and field recordings as the sources for The Foley Room (2007), without basically changing style…

Tobin’s studies on timbre should also not be overlooked. The apparently unassuming “Defocus” is actually a new kind of symphony. Tobin warps the distinctive tone of an instrument to produce a new kind of instrument, and then weaves a few of them (a bee-like violin, a distorted bass, UFO-sounding flutes) into an organic flow of sound. It is, in fact, one of the most significant innovations since Beethoven added a choir to a symphony.

Needless to say, jazz fuels and dresses these compositions. However, Tobin does to jazz what Picasso did to impressionism: it uses only discrete fragments of the image to reconstruct the whole. Furthermore, it is never the only or main element. For example, the sax solo of “Wires And Snakes” coexists with industrial metronomic pulses and with soothing ambient waves of electronics.

[Read more…]

Media diet for Q2 2020

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Okay, music I most enjoyed discovering this quarter:

[Read more…]

Media I’m looking forward to, Q3 2020 edition

Added this quarter:

Books

bold = especially excited

[Read more…]

Media diet for Q1 2020

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Okay, music I most enjoyed discovering this quarter:

[Read more…]

Media I’m looking forward to, Q2 2020 edition

Added this quarter:

Books

bold = especially excited

[Read more…]

Funny or interesting Scaruffi Quotes (part 5)

Previously: 123, 4.

On Primus:

A stubbornly alternative group, alien to the commercial route, immune to the lure of compromises, heir of the “freak” philosophy and ethics, and representative of the genealogical line of “neo-freaks” inaugurated by the Butthole Surfers – that’s Primus. Created by bassist and vocalist Les Claypool, Primus was a bright spot among the rock groups of the early 90’s. Each track was like a stylistic puzzle; the group had few predecessors as their style resembled progressive-rock (from Frank Zappa to Rush) but had the feel of hard-core. Listeners can hear echoes of Minutemen and Black Flag, but the smooth progression between tones was anything but punk.

On Rake:

Their first album… contains two lengthy improvisations… driven by a jazz guitarist who listened to John McLaughlin till he went nuts and by a keyboardist who fell in love with the Moog. The sound is an aberration of Albert Ayler and Borbetomagus.

…The first CD [of their 2nd album] contains four lengthy suites… The second [CD] contains 75 brief pieces, whose dementia reaches disturbing levels; a wild collage of abstract sonic miniatures that rarely coalesce in songs. The 4th is a masterpiece of punk-rock, the 11th and the 21st are masterpieces of avantgarde guitar, the 55th and following ones are space-rock at its best, the 64th and following ones are gothic/ambient psychedelia, the 73th and following ones are the childish conclusions of the whole big nonsense. A totally pointless genius, as Dada would have loved.

[Read more…]

Initial observations from my 2nd tour of rock history

I’m still listening through Scaruffi’s rock history, building my rock snob playlist as I go. A few observations so far:

  1. Relative to last time I listened through Scaruffi’s rock history (>8 years ago IIRC), my tastes have evolved quite a lot. I notice I’m more quickly bored by most forms of pop, punk, and heavy metal than I used to be. The genre I now seem to most reliably enjoy is the experimental end of prog-rock (e.g. avant-prog, zeuhl). I also enjoy jazz-influenced rock a lot more this time, presumably in part because I listened through Scaruffi’s jazz history (and made this guide) a couple years ago.
  2. I am more convinced than ever that tons of great musical ideas, even just within the “rock” paradigm, have never been explored. I’m constantly noticing things like “Oh, you know what’d be awesome? If somebody mixed the rhythm section of A with the suite structure of B and the production approach of C.” And because my listen through rock history has been so thorough this time (including thousands of artists not included in Scaruffi’s history), I’m more confident than ever that those ideas simply have never been attempted. It’s been a similar experience to studying a wide variety of scientific fields: the more topics and subtopics you study, the more you realize that the “surface area” between current scientific knowledge and what is currently unknown is even larger than you could have seen before.
  3. I still usually dislike “death growl” singing, traditional opera singing, and most rapping. I wish there were more “instrumental only” releases for these genres so I could have a shot at enjoying them.
  4. Spotify’s catalogue is very choppy. E.g. Spotify seems to have most of the albums from chapter 4.12 of Scaruffi’s history, and very few albums from chapter 4.13. (I assume this is also true for iTunes and other streaming providers.)

Media diet for Q4 2019

Music

Spotify playlist for this quarter is here. Playlists for past quarters and years here.

Okay, music I most enjoyed discovering this quarter:

[Read more…]