I’ve seen seven main viewpoints on AI and the future from those who spend a lot of time thinking about it:

(1) Superintelligence Doomers – they believe we are likely to build AI that’s superintelligent (i.e., that surpasses human intelligence in all respects) and that once we do, it will kill or enslave humanity.

See: Eliezer Yudkowsky

“The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else.”

(2) AI Corrosionists – they believe AI, while it could be beneficial, has a very substantial likelihood of making things far worse. Unlike doomers, they see most risk coming from other processes rather than a sudden AI takeover or annihilation: for instance, it could be that humans lose control over the future slowly (by ceding more and more control over to AIs that are optimizing for things that are different than human values), or it could be that AI de-stabilized aspects of society that make other large risks (like nuclear war) more likely

See: Paul Christiano

“If you imagine a society in which almost all of the work is being done by these inhuman systems who want something that’s significantly at cross purposes, it’s possible to have social arrangements in which their desires are thwarted, but you’ve kind of set up a really bad position. And I think the best guess would be that what happens will not be what the humans want to happen, but what the systems who greatly outnumber us want to happen.”

(3) Near Risk Doomers – they believe AI will have very bad effects (e.g., increasing unfairness, authoritarianism, climate change, unemployment, or concentration of power) but that we’re not going to build superintelligence anytime soon.

See: Cathy O’Neil

“The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer.”

(4) Current Paradigm Doubter – they believe the current AI paradigm is overhyped and isn’t going to change things all that much (some of them see the current paradigm as net negative; others see it as positive but only as one useful and overhyped tech among many other useful technologies). Some, like Marcus, hope that future paradigms might be more trustworthy and beneficial.

See: Gary Marcus

“What a strange world… All the major AI companies spending billions producing almost exactly the same results using almost exactly the same data using almost exactly the same technology, all flawed in almost exactly the same ways. Historians gonna be scratching their heads…The only way we will move significantly forward is to develop new architectures—likely neurosymbolic—that are less tied to the idiosyncrasies of specific training set.”

(5) AI Stewardists – they believe AI advancement is important to humanity’s future, but it’s not necessarily going to be good or bad: it will be what we make of it, so it should be developed thoughtfully based on what we want to achieve.

See: Kevin Kelly

“AI could just as well stand for ‘alien intelligence.’ We have no certainty we’ll contact extraterrestrial beings in the next 200 years, but we have almost 100 percent certainty that we’ll manufacture an alien intelligence by then. When we face these synthetic aliens, we’ll encounter the same benefits and challenges that we expect from contact with ET. They will force us to reevaluate our roles, our beliefs, our goals, our identity. What are humans for?”

(6) Near Benefit Boosters – they believe AI will be very useful, impactful and important as a technology, and they also believe superintelligence is a ridiculous thing to worry about.

See: Yann LeCun

“AI is intrinsically good, because the effect of AI is to make people smarter….AI is an amplifier of human intelligence and when people are smarter, better things happen: people are more productive, happier and the economy thrives.”

(7) Superintelligence Boosters – they believe we are likely to build superintelligent AI and that it will usher in an incredible and positive new era.

See: Ray Kurzweil

“By the time of the Singularity, there won’t be a distinction between humans and technology. This is not because humans will have become what we think of as machines today, but rather machines will have progressed to be like humans and beyond. Technology will be the metaphorical opposable thumb that enables our next step in evolution.”

How do we best organize the positions on risks from AI?

I think the two most informative spectrums to think about are

(1) How *substantial* the near-term impacts of AI are expected to be

and

(2) Whether the effects of AI are likely to be *good* or *bad*

The image shown is my attempt to place these different thinkers who talk about AI on this two-axis system. Though note that this just reflects my best guesses, some of these thinkers have very nuanced views, and I’m not certain that I’m placing each of them in the right place.

This piece was first written on July 27, 2024, and first appeared on my website on August 4, 2024.

When people are in a situation where they’re as rewarded for doing an easier, fake version of something as they are for doing the real, valuable version, you’ll usually find that most of the activity is fake.

There are two types of “fake” activity in the way I’m using the word:

(1) Activity that doesn’t achieve anything useful at all.

Example: a treatment that claims to cure a disease but doesn’t actually help the patients who use it.

(2) Activity that claims (and is widely believed to be) about X but is really about Y.

Example: the FDA says that it won’t approve some European sunscreens because of a lack of sufficient safety data. Some people (including my dermatologist!) claim – and I don’t know whether they are right – that the real reason is that the FDA doesn’t want the difficult challenge of standing up to entrenched sunscreen interests that would lose out from allowing these newer sunscreens.

This theory can be further extended to Delusional Replica Theory:

Most (but not all) of those doing the fake activity won’t really believe they are producing B.S. and will find arguments (to convince themselves and others) for why their activity isn’t fake. However, on some level, they may still have doubts about the authenticity of what they’re doing.

Here are a few snarky applications of Delusional Replica Theory:

1. Some major parts of the education sector ignore the science of learning and don’t seem to have figured out what the goals of education should be. Yet you don’t need to teach valuable skills/information to have a successful school or to be a successful teacher.

(Hint: the goals shouldn’t be to memorize things that you are bound to soon forget, nor should they be to give mostly useless knowledge in exchange for a decade of debt.)

2. A substantial portion of the crypto industry fails to provide the value claimed. Yet people buy anyway.

(Why have a regular company with a regular database when a blockchain achieves the same thing much less effectively?)

3. Some parts of the finance industry offer negative value services. Yet people buy them because they mistake them for value-added services.

(Want a loan for a product that won’t make your life any better, an extremely risky stock option you don’t understand, or an expensive way to invest in the stock market?)

4. Many multi-level marketing (MLM) companies extract money from their own contractors. But they manage to get numerous people to sign up by showing examples of the few who have made lots of money and by frequent allusions to how their offerings can help you achieve your dreams.

(Usually, people who work for a company get paid by the company, not the other way around! For most MLM companies, a shockingly low percentage of those involved make a decent amount of money.)

5. Many supplement companies provide pills that don’t do what they claim and sometimes that don’t even contain the dosages or ingredients they claim. But they manage to sell them anyway since it’s hard to tell if a supplement is working.

(Are you in the market for a slightly dangerous placebo?)

6. A non-negligible percent of social science can’t seem to be replicated using the same study design on a similar population. But since reviewers don’t know that a result is not going to be replicable, unreplicable results still get published. 

(I estimate this at ~40% of papers in top journals.)

I’d be interested to know: what is a real-world example you’ve seen where Replica Theory may apply? That is, what’s an area where you think much of the activity is fake due to people being rewarded for doing a fake, easier version of activity as much as they are rewarded for doing the real thing?

This essay was first written on October 27, 2021, and first appeared on this site on February 18, 2022.