I’ve seen seven main viewpoints on AI and the future from those who spend a lot of time thinking about it:

(1) Superintelligence Doomers – they believe we are likely to build AI that’s superintelligent (i.e., that surpasses human intelligence in all respects) and that once we do, it will kill or enslave humanity.

See: Eliezer Yudkowsky

“The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else.”

(2) AI Corrosionists – they believe AI, while it could be beneficial, has a very substantial likelihood of making things far worse. Unlike doomers, they see most risk coming from other processes rather than a sudden AI takeover or annihilation: for instance, it could be that humans lose control over the future slowly (by ceding more and more control over to AIs that are optimizing for things that are different than human values), or it could be that AI de-stabilized aspects of society that make other large risks (like nuclear war) more likely

See: Paul Christiano

“If you imagine a society in which almost all of the work is being done by these inhuman systems who want something that’s significantly at cross purposes, it’s possible to have social arrangements in which their desires are thwarted, but you’ve kind of set up a really bad position. And I think the best guess would be that what happens will not be what the humans want to happen, but what the systems who greatly outnumber us want to happen.”

(3) Near Risk Doomers – they believe AI will have very bad effects (e.g., increasing unfairness, authoritarianism, climate change, unemployment, or concentration of power) but that we’re not going to build superintelligence anytime soon.

See: Cathy O’Neil

“The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer.”

(4) Current Paradigm Doubter – they believe the current AI paradigm is overhyped and isn’t going to change things all that much (some of them see the current paradigm as net negative; others see it as positive but only as one useful and overhyped tech among many other useful technologies). Some, like Marcus, hope that future paradigms might be more trustworthy and beneficial.

See: Gary Marcus

“What a strange world… All the major AI companies spending billions producing almost exactly the same results using almost exactly the same data using almost exactly the same technology, all flawed in almost exactly the same ways. Historians gonna be scratching their heads…The only way we will move significantly forward is to develop new architectures—likely neurosymbolic—that are less tied to the idiosyncrasies of specific training set.”

(5) AI Stewardists – they believe AI advancement is important to humanity’s future, but it’s not necessarily going to be good or bad: it will be what we make of it, so it should be developed thoughtfully based on what we want to achieve.

See: Kevin Kelly

“AI could just as well stand for ‘alien intelligence.’ We have no certainty we’ll contact extraterrestrial beings in the next 200 years, but we have almost 100 percent certainty that we’ll manufacture an alien intelligence by then. When we face these synthetic aliens, we’ll encounter the same benefits and challenges that we expect from contact with ET. They will force us to reevaluate our roles, our beliefs, our goals, our identity. What are humans for?”

(6) Near Benefit Boosters – they believe AI will be very useful, impactful and important as a technology, and they also believe superintelligence is a ridiculous thing to worry about.

See: Yann LeCun

“AI is intrinsically good, because the effect of AI is to make people smarter….AI is an amplifier of human intelligence and when people are smarter, better things happen: people are more productive, happier and the economy thrives.”

(7) Superintelligence Boosters – they believe we are likely to build superintelligent AI and that it will usher in an incredible and positive new era.

See: Ray Kurzweil

“By the time of the Singularity, there won’t be a distinction between humans and technology. This is not because humans will have become what we think of as machines today, but rather machines will have progressed to be like humans and beyond. Technology will be the metaphorical opposable thumb that enables our next step in evolution.”

How do we best organize the positions on risks from AI?

I think the two most informative spectrums to think about are

(1) How *substantial* the near-term impacts of AI are expected to be

and

(2) Whether the effects of AI are likely to be *good* or *bad*

The image shown is my attempt to place these different thinkers who talk about AI on this two-axis system. Though note that this just reflects my best guesses, some of these thinkers have very nuanced views, and I’m not certain that I’m placing each of them in the right place.

This piece was first written on July 27, 2024, and first appeared on my website on August 4, 2024.

Thanks to William MacAskill’s excellent new book on the topic (What We Owe the Future), lots of people are talking about longtermism right now. For those not familiar with the concept, “longtermism” is the ethical view that “positively influencing the long-term future should be a key moral priority of our time.”

Below are some of my favorite arguments for longtermism, followed by some of my favorite against it. Note that I borrow from Will’s book heavily here in the section on arguments in favor of longtermism.

Five of my favorite arguments FOR longtermism:

1. We value future people. If someone creates a hidden nuke set to explode in 150 years, we can agree this is highly immoral, even if we’re sure none of the people harmed exist today. And we can agree it’s wrong to neglect the world so much that it’s a hell hole for our grandchildren.

2. There may be VAST numbers of future people. We can agree that harming more people is worse than harming fewer and that helping more people is better than helping fewer. So if our choices affect all future people, that puts a large moral weight on those actions.

3. We may live at a pivotal time to influence the future. Because of technologies that might cause existential catastrophes for humanity (e.g., nuclear bombs, human-engineered viruses, advanced A.I.), our choices today could have a massive impact on the future of our species.

4. It’s neglected – relatively little work goes into improving the far future. News cycles, political terms, profitability requirements, and many other factors push us towards thinking near-term. It seems likely that less than 1% of current resources go into trying to improve the long-term future.

5. Future people don’t have a say. When we take actions impacting the environment or our chance of extinction or technological development, we do so without input from the future people that may be affected. Historically, groups with no say are often ethically undervalued.

Six of my favorite arguments AGAINST prioritizing longtermism:

1. The future may not be big. Much of the pull of longtermism comes from assuming the future could have FAR more people than alive today. But if there’s a lower bound on the yearly probability of extinction that you can’t change, the expected length of the future may not be long.

2. Humans need tight feedback loops. My opinion is that without such feedback toward a clear objective, human behavior usually spirals off into something weird (e.g., impracticality or status signaling). It can be VERY hard to tell if we’re positively influencing the long-term.

3. There may be valid reasons to discount the future. Empirically, few people care about far future people nearly as much as those alive today (the further out, the less they care). If there’s no such thing as objective moral truth (a debatable question), they aren’t making a mistake.

4. Longtermism is less compelling than other arguments for similar goals. Most people really care about humanity not going extinct and want the world to be a good place for our grandchildren (even if they don’t now have grandchildren). Bringing in considerations of humans in 20000AD makes these important topics less convincing.

5. YOUR probability of creating far future positive change may be tiny. Longtermism says if YOU have a 1 in a million chance of non-negligible far future influence, that’s incredibly valuable. And more generally, it can be very hard to figure out what to do now that will positively influence the world (with any significant degree of confidence) hundreds or thousands of years from now. I think it may be wiser to focus on a higher probability of, for instance, preventing calamity in 30 years (rather than, say, focusing on preventing it from occurring in the far distant future).

6. It’s debatable whether it’s better for more people to exist. Philosophers don’t agree whether a world with vastly more people is better than one with fewer (see discussions of “population ethics”). I think that most lay people also prefer a world with billions of very happy people to one with quadrillions of very slightly happy people (even if the latter sums up to far more total happiness). While we are not necessarily forced to choose between these options (maybe we could have a world of a huge number of very happy people), it’s not obvious we should be aiming for a vast future rather than one with very high levels of good things (e.g., only tens of billions of people but where everyone has very high well-being, freedom, self-actualization, etc.).

This essay was first written on August 30, 2022, and first appeared on this site on September 1, 2022.

Here’s a list of some of those theories (as I interpret them), explained in extremely concise terms as best as I could manage:

The List of Thinkers and Their Core Ideas:

1. Peter Singer

We would ruin expensive shoes to save a drowning child we walked by, but not forgo expensive shoes to save one far away. We say it’s wrong to torture animals, yet our factory farms constantly torture them to make the meat/eggs we choose to buy. We should do more good.

2. Nick Bostrom

Technology is often good, but not always, and sometimes it is incredibly dangerous (e.g., nuclear weapons, engineering of viruses). If we invent enough technologies, eventually we may invent one that totally devastates civilization, even if actors are well-intentioned.

3. Eliezer Yudkowsky

Each year, AI gets more powerful. It’s on course to eventually be smarter than the smartest human in almost all ways. But these AIs are alien minds that won’t embody what we care about sufficiently well. When that happens, it will likely go extremely badly for humanity. Nobody knows how to make such a system safe.

4. Robin Hanson

Many of the explanations for why we do things don’t make sense (if we did X for claimed reason Y, we’d also do Z, which we don’t do). The most parsimonious explanation is that much of our behavior is just social signaling, which we engage in without even realizing it.

5. Elon Musk

If we let climate change or A.I. get out of control before we get off this planet, we’re F’d. Let’s get off this planet before it’s too late.

6. Jonathan Haidt

We often think we use our reason to arrive at moral conclusions, but mainly we use our reason to try to rationalize or justify what our moral intuition tells us. Political parties/groups have differing moral intuitions, and there are good people on all sides.

7. Noam Chomsky

We view the U.S. as a democracy, yet large corporations and a small group of elites have far more sway over what happens than anyone else. We view the U.S. as a world benefactor, yet its actions abroad are self-serving.

8. Andrew Yang

Automation is eventually going to take away a massive number of jobs (with e.g., self-driving cars and trucks, automated checkout, and AI assistants). We need to figure out how to make society work for all as jobs disappear and wealth becomes even more unequally distributed.

Of course, I had to vastly oversimplify your views to fit them in this format – hopefully I did so about as accurately as the format allows.

This piece was first written on August 26, 2020, and first appeared on my website on May 18, 2026.