But there is also something soul-crushing about it. People spend decades learning a craft and then see an A.I. make something in a few seconds or minutes that other people see as comparable to their own work.
And it will be widely misused: to create ripoffs of copyrighted works without giving credit, to generate billions of pages of low-quality content in order to game search engines, to make hyper-personalized spam and commit phishing and fraud, to mislead the public with misinformation and customized persuasion and outrage bait, and to create even more intense social media and video addiction.

Then, there are the longer-term consequences of this kind of technology, which are hard to predict but may be truly catastrophic (even according to some of the leaders who run these very companies).

Humanity sometimes creeps along cautiously – as with the development of new medicines, where we (arguable) allow millions to die based on the fear of releasing something unsafe (which is, to a meaningful extent, a legitimate fear – getting the tradeoffs right is hard). And in the case of nuclear power, where regulation impedes its development due to (mostly) ungrounded fears.

Yet with A.I., humanity plunges forward into the abyss at full speed, with no breaks, with the train conductors announcing to the passengers that we can’t predict where the trains are going, that this all may end up a disaster, perhaps cataclysm, as they shovel exponentially more coal into their white-hot furnaces.

This piece was first written on April 11, 2024, and first appeared on my website on April 16, 2024.

But I think approximately everyone has at least one of those beliefs; it’s just not that obvious because many of these strange beliefs are idiosyncratic, and people often hide them from each other and sometimes even from themselves (by avoiding thinking about that belief, or by not reflecting on how crazy the belief would seem to others).

For one person, the weird belief may be their lucky coin (see this lovely essay that I learned about from @mariellevolz after writing a first draft of this essay).

For another person, it may be an unsupported belief that they are unlovable.

For a third person, it might be an attachment to a certain diet that they’re convinced would solve all sorts of problems if everyone would just go on it (like they did).

Or it could be a strange belief they learned from their childhood religion, which everyone except adherents of their exact denomination of their religion sees as ridiculous.

And so on.

Because I think just about everyone has strange false beliefs like this, I try not to judge people too much based on the fact that they have a “weird” belief that I think is way out of alignment with reality. Of course, even if I’m right, that doesn’t mean everyone is an equally “bad” thinker.

Some people think, “That person believes X, what an idiot,” and write the person off. I aspire not to do that based on one crazy belief but rather to look at a person’s beliefs and how they arrive at their beliefs more holistically.

Being an effective thinker is better measured by how we do, on average, on the important things (weighted by their importance) rather than the worst thinking we do on any single topic.

I also believe that with time and consistent effort, we all can get better at thinking clearly and that it’s a worthwhile investment for most people.

But even with hard work, we’re unlikely to get to zero weird false beliefs.

This piece was first written on February 2, 2024, and first appeared on my website on March 7, 2024.

Below is my list of what I see as the deepest mysteries. 

What would you add to the list?

—

LIST OF DEEP MYSTERIES ABOUT THE NATURE OF REALITY

—

1. THE UNIVERSE

1.1 Eternity – Will our universe last forever? If it won’t, what will the end of the universe be like (e.g., a new big bang, a big crunch, time simply ending, something else)?

1.2 Genesis – Did our universe have a “starting point” (in a pre-existing expanse of time) or has it existed forever, or did time begin when the universe began?

1.3 Geometry – Does the universe extend infinitely in all directions, or is it finite? (To get a feel for how it might be finite without requiring a boundary, consider that the universe could be like the game Pac-Man, topologically – in the sense that, traveling long enough in any direction, you eventually get back to where you started). Is empty space perfectly flat (e.g., like an infinite plane but in multiple dimensions), or does the universe have implicit curvature (e.g., like the surface of a universe-sized orange)?

1.4 Constraints – Could the laws of physics be any different than they are, or is there some reason they have to be this way? Why are there the particular elementary particle types that we find (e.g. electrons, quarks, neutrinos, etc.) with the particular properties they have? Why are there the particular forces we find (e.g. electromagnetism, gravity, the strong nuclear force, etc.) with the particular properties they have?

1.5 Entropy – Why was our universe in a low entropy state in the distant past (with the matter fairly uniformly spread out, rather than, for example, all condensed into black holes)?

1.6 Reason – What’s the right explanation for WHY our universe came to exist? If it was not created by any form of intelligence, is the question even coherent to ask? If it was created by some form of intelligence, was it God, or some other form of intelligent beings (e.g. aliens creating a universe simulation in some vast computer)? If there was an intelligent creator, what is the nature of that creator (e.g. what is that creator like, what does that creator care about, etc.)?

1.7 Uniqueness – Are there other universes besides our own?

1.8 Travel – Are wormholes (connecting different parts of spacetime) actually possible? Is time travel forbidden by the laws of physics? Is instant teleportation impossible?

1.9 Dimensionality – How many dimensions of spacetime are there? (e.g., the standard view is four dimensions, which come from three spatial dimensions and a single time dimension, whereas ‘string theory’ suggests more, though most of those extra dimensions are believed to be really tiny/compact)

1.10 Computability – Can the universe be simulated to arbitrary accuracy on a normal computer (given sufficiently large amounts of memory and time), or is there something “incomputable” about the universe?

1.11 Expansion – Why does the universe seem to be expanding at an accelerating rate? If dark energy exists (the hypothesized element that is believed to make up most of the energy of the universe, not to be confused with dark matter), then what is it and why is it there?

1.12 Dark Matter – Does dark matter actually exist (i.e., the hard to detect element that appears to make up most of the mass in the universe)? If so, what is it made of?

1.13 Unification – How can general relativity and quantum mechanics be combined into a consistent theory that generalizes both (e.g., to model what happens when extremely tiny things are in strong gravitational fields)? Is ‘string theory’ the right path towards unifying general relativity and quantum mechanics (as many physicists believe), or is it a misguided approach (as a few physicists argue)?

1.14 Energy – Is there a positive, zero, or negative total amount of energy in the universe?

1.15 Divisibility – Are space and time infinitely divisible, or are there truly minimum lengths and durations?

1.16 Aliens – Does intelligent life (or even non-intelligent life) exist elsewhere in the universe? If intelligent life does exist elsewhere in the universe, why does it appear not to have reached us yet? Will we ever encounter life that has non-earth origins?

1.17 Life – Why does our universe have a set of physical laws and physical constants that allow for life (and consciousness) to exist at all? Some people argue that if the strengths of the basic forces of physics (like gravity) had been more than a certain increment stronger, or more than a certain increment weaker, life could not have formed in our universe.

—

2. HUMANITY

2.1 Origin – What were the first “replicating entities” that all the humans today eventually developed from? Many religions say that God placed the first humans here, who eventually gave birth to all the others. But as for non-religious (evolutionary) explanations, there are various theories about what these replicating entities might have been (e.g., crystals, or special molecules that can make other molecules they bump into look like themselves)

2.2 Extinction – When will humanity go extinct? And what will cause humanity’s extinction?

2.3 Governance – Given the flaws and limitations of our species, and our current state of technology, what systems of governance, laws, and institutions would maximize human flourishing?

2.4 Happiness – Given the current state of the world, and the nature of and resources of one particular person, if that person wants to maximize their happiness, what should they do? What would a supremely intelligent being tell a human about how to become happier?

2.5 Qualia – Do humans have different internal experiences in cases where we typically assume them to have the same experience? For example, do there exist non-visually impaired people whose internal experiences of red (e.g., when looking at a red apple) are totally different than each other? For instance, could it be that one person’s experience of red is what another person experiences as blue? Or even that one person experiences reds as being somewhat more like what someone else experiences for blue things?

2.6 Humor – Why do humans have humor? There are many theories (e.g., “benign violation theory” and “superiority theory”) but none of them seem complete/comprehensive.

2.7 Music – Why do humans love music so much? It’s hard to understand this from an evolutionary perspective.

2.8 Yawning – Why do we yawn? And why are yawns contagious (i.e., seeing someone yawning tends to make others yawn)? There are various theories (e.g., to cool down the brain, or to get more oxygen). My preferred highly speculative explanation is that it’s a mechanism for groups to sync their sleeping, but it’s really hard to know if that’s right.

2.9 Nutrition – What should we do to not merely stave off malnutrition, but to thrive and be as healthy as possible? There seems to be a disturbing lack of consensus on this question (and it may turn out to depend a lot on individual people’s genetic and behavioral differences).

—

3. KNOWLEDGE

3.1 Induction – is there a sound argument in favor of using induction that is non-circular (i.e., that doesn’t implicitly rely on using induction to make the argument)?

3.2 Occam’s Razor – Is there a sound justification for Occam’s razor (by which I mean the claim that “simpler” explanations are more likely to be true). If so, what’s the right notion of simpleness for a hypothesis that doesn’t require making arbitrary choices (e.g., avoiding the issue Kolmogorov complexity has where it introduces an arbitrary choice of representation language)?

3.3 Cognition – What algorithms are human brains running that allow humans to learn, remember, model the future, model other minds, reason, plan, theorize, and make inferences? Is it possible to build something broadly as smart as (or much smarter than) a human using just incremental improvements on top of today’s deep learning algorithms (combined with larger data sets and faster computation)?

3.4 Infinities – How should we think about maximizing the expected value of an action in contexts where we can’t assign strictly zero probability to outcomes of infinite value? The mere possibility (i.e. non-zero probability) of an infinite value seems to mess up the calculations completely. If we’re trying to maximize expected value, how do we resolve “pascal’s wager” and “pascal’s mugging” type situations?

3.5 Priors – How can we assign prior probabilities to hypotheses in a principled, computable way (that is, how do we decide what probabilities to assign to hypotheses BEFORE we are given evidence to use to update those prior probabilities)?

3.6 Anthropics – How do we perform reasoning and probabilistic estimation in “anthropic” scenarios where we are forced to consider the probability of even being the sort of mind that could end up in that scenario? What’s the right way to think about questions like “what’s the probability that I would end up being me rather than someone else?” or “what’s the probability that I would end up being a human rather than another species?” or “what’s the probability that I would end up being one of the last 1 billion people to be born before humanity goes extinct?”

3.7 Reference Classes – If we have multiple categories something falls into (each implying that we make different predictions about that things), how do we combine this information into a final prediction? For instance, if ALL we know is that X is a flying car, and flying machines rarely have property Y, yet cars usually have property Y, how do we make a principled “best” estimate of the chance that X has property Y? In other words, what’s the right way to think about combining the information Prob( Y, given A ) with Prob( Y, given B ) when what we really need to know is Prob ( Y, given A and B )?

3.8 Black swans – How do we probabilistically model situations (or do expected value calculations) when we know that “black swan” events (that are unlike any we have seen in the past) are possible, even though we don’t (by definition) know what these events will actually be like and how likely they are to occur?

3.9 Consciousness – Can we be 100% certain that some form of consciousness exists (because we have direct perception of conscious experiences), or should we be less than totally certain even about this?

—

4. EXISTENCE

4.1 Many Worlds – Are the “many worlds” of quantum mechanics actually all literally existing (i.e., are they as real as what we all are experiencing right now)? Or does the mathematics just make it seem that way? If they are not really there, what’s the resolution to the “measurement problem” in quantum mechanics (e.g., how do we define what a measurement is and is not such that we have a complete description of when quantum wave functions collapse)?

4.2 Anything – Why does “something” exist, rather than there being nothing at all? Or does this question not even make sense to ask?

4.3 Time – Is there a meaningful sense in which all times that have and will happen exist at once, or do some times only come into existence (as time passes)? Note that the theory of relativity seems to undermine the possibility of a single-speed of time that is the same for all observers.

4.4 Morality – Is there any form of morality that is “objectively” correct? For instance, can moral statements like “murder is always wrong” be true or false in the way that “I once purchased a fedora” is either true or false? If any sort of objective moral truth is possible, what then is objectively true about morality (e.g., utilitarianism, the categorical imperative, virtue ethics, theological ethics, etc.)?

4.5 Non-physical – does anything exist that is not merely made of atoms / not bound by our laws of physics, that can directly cause changes in our world (e.g. souls, ghosts, gods, spirits, the devil, etc.)?

4.6 Time travel – is time travel prevented by the laws of the universe? If it is not prevented by the laws of the universe, has it or will it ever happen? Potentially related: is it possible to exceed the speed of light, or is that literally impossible (as our current theories seem to tell us)?

—

5. CONSCIOUSNESS

[Note that, by “consciousness,” I mean the state of having “internal experiences.” A being has consciousness if there is “something that it’s like” to be that being. For instance, there is something that it’s like to be you, but not something that it’s like to be a chair. You have internal experiences, like experiencing the taste of a pineapple or the color of a red apple, but a chair has no experiences. You have consciousness, in the way I’m using the term, but a chair (almost certainly) doesn’t.]

5.1 Justification – Why is there consciousness at all? Couldn’t the universe be just the same as it is now except without any internal experiences at all (i.e., with no consciousness)?

5.2 Requirements – What sort of configurations of matter are necessary to give rise to consciousness? Would it be possible to build a physical device to measure consciousness? If so, what would such a device need to be like?

5.3 Physics – How do we unify the existence of consciousness with the currently known laws of physics (since examining our known laws of physics would not allow you to infer that consciousness experiences even occur)?

5.4 Quantum – Does the human brain exploit quantum physics in a meaningful way, such that it is hard to understand what the brain is doing without using a quantum mechanical explanation?

5.5 Free Will – Why do we have the persistent sense of having free will, even though (given our current understanding of physics) our actions are fully and completely determined by whatever happened a moment before (plus quantum uncertainty)?

5.6 Minimal – Which beings have consciousness? Do atoms? Viruses? Bacteria? Blood cells? Lice? Ladybugs? Spiders? Snails? Frogs? Mice? Beavers? Toucans? What’s the “simplest” possible brain or system or algorithm that can experience consciousness?

5.7 Evolution – Did consciousness come about as a result of evolution (i.e., was it created by selection pressures), and if so, what is its survival function exactly?

5.8 Algorithmic – Is it possible for an algorithm run on a digital computer to experience consciousness?

5.9 Intelligence – Can something be much more intelligent than human beings (broadly speaking) and not have consciousness? Can something behave just like a human in all ways relevant to intelligence and yet not experience consciousness (i.e., can “philosophical zombies” exist?)

5.10 Teleportation – If a teleportation device existed that could make an essentially perfectly accurate copy of you out of new atoms, with all your memories and personality intact, but it destroyed your original self just before assembling the new copy of you, would the copy be you, in the same sense that you one second from now is still you?

5.11 Macro – Is it possible (even if extraordinary difficult) for a large-scale, purely mechanical system to have consciousness, for instance, a massive machine made out of gears and pulleys? Could a very large number of people, if they were all carrying out coordinated movements that were designed to match the algorithmic information processing of a brain, temporarily create a large-scale consciousness?

5.12 Agent – What’s the right definition to use for a single “being” or “agent” that properly distinguishes it from all other beings, while still handling even weird thought experiments. (For instance, where a person’s brain is split into two but continues to operate, or where a brain is slowly replaced with pieces of another brain over a long period of time, without ever ceasing operation)?

5.13 Finality – Does all experience cease after death, as atheists typically believe and as spiritual and religious people typically deny?

What is the scope of morality?

If we look across cultures (including micro-cultures that exist within other cultures), there is a vast number of things that people view as immoral. However, if you eliminate those that are only viewed as immoral because they are believed to lead to other things viewed as bad, the list becomes a lot smaller.

So, what are those things that at least some human cultures view as inherently immoral, that is, acts they would still consider immoral even if no other consequences of that behavior were to occur?

Here’s my attempt at a comprehensive list of things that at least some cultures view as intrinsically wrong. Note that many of these items are related or overlapping.

UNFAIRNESS

• Injustice (e.g., helping a bad person avoid punishment or undermining someone’s just reward)
• Inequality (e.g., causing societies’ resources to be concentrated among just a few while everyone else is poor, or treating people differently based on gender or skin color)
• Abuse of authority (e.g., nepotism, or favoritism, or those in power giving rewards to those they like best instead of those who deserve it, or those who are given a certain position of authority not carrying out the duties of that position, or carrying that the duties poorly or in a self-interested manner)

HARM

• Harm (e.g., causing others suffering or purposely reducing the happiness of others)
• Murder (e.g., ending the life of another person, including in some cultures, animals, or allowing others to die needlessly)
• Genocide (e.g., the harming or killing of a specific, targeted group)
• Stealing (e.g., taking something that belongs to someone else or that should be no one’s property)
• Destruction (e.g., damaging or annihilating things of value, like ruining the environment, destroying ancient or beautiful artifacts, instigating societal collapse, or hunting a species to extinction)
• Slander (e.g., speaking ill of others or gossiping by spreading true but negative or harmful information about other people)
• Subjugation (e.g., controlling the body, mind, or choices of others, or restricting freedom)

DISRESPECT

• Disrespect of authority (e.g., being rude to your parents or disobeying your leaders)
• Disrespect of the vulnerable (e.g., treating people badly or not showing respect for members of a subjugated or vulnerable or oppressed group, or not paying proper respect to those who are victims, or disrespecting people living under hard conditions)
• Disrespect of the dead (e.g., digging up a grave, selling a dead body, necrophilia, cannibalism, speaking badly of the dead)
• Disrespect of god or gods (e.g., blasphemy, taking god’s name in vain, desecrating a place of worship, violating a commandment such as doing work on a day when work is forbidden, worshiping a carved idol instead of worshiping God, not engaging in prayer or thanks, questioning gods nature, or gods choices, or gods existence)
• Practicing other faiths (e.g., worshipping alternative gods, engaging in superstitions, engaging in practices or rituals of other religions)
• Rule-breaking (e.g., violating the laws of your country or the rules of your culture even if those laws are arbitrary or pointless)
• Violating tradition (e.g., non-conformity, or refusing to engage in the traditions of your culture, or flouting traditional roles or norms)
• Disrespect of opinions (e.g., not taking into account the opinions or desires of other people, falsely believing you know more or are wiser than others)
• Defiling (e.g., causing others to be impure, or causing others to take)

How big does P need to be before it’s irresponsible to say, “Well, the horrible event is unlikely to happen, so let’s not worry about it”?

50% (1 in 2)?
5% (1 in 20)?
1% (1 in 100)?
0.1% (1 in 1,000)?
0.01%? (1 in 10,000)?
0.001%? (1 in 100,000)?
0.0001%? (1 in 1,000,000)?

To me, it seems bizarre to dismiss a threat that has even a 1 in 1000 chance of causing the worst events in human history (over the next 50 years) with an argument like “I think it’s unlikely to happen; therefore, let’s stop worrying about it.” Most people (quite rationally) decide it’s worth wearing a seatbelt with each drive, despite far less than a 1 in 1000 chance of a car accident on each trip. Yet, as a society, we often collectively go without our metaphorical seatbelts when it comes to societal risks. And with societal risks, unlike with seatbelts, we often can’t even really say what the probability of these risks is – some people have an intuition that they are very small, but many others believe the risk is high.
 
Of course, preparing for risks doesn’t mean we should stop everything else in the meantime; it just means that we should take collective action to reduce these risks as long as the cost per unit of risk reduction is not too high. And yet, we often fail to do so.

There are a number of activities humanity is doing right now that one could reasonably argue have a P of at least 1 in 1000 and perhaps much higher than that. For instance:

Our lack of strong controls on pathogen research enables the potential creation of engineered viruses or bacteria that could lead to terrible pandemics.

Our lack of safeguards on AI development leaves open the possibility of super-intelligent AI that we are unable to control or super-intelligent AI that gives one group unreasonable levels of power.

Our failure to create global coordination around climate change leaves open the possibility of catastrophic climate impacts.

The most popular argument given for ignoring these potential risks is that they seem (to the person who advocates ignoring them) unlikely to happen. While many disagree that these are unlikely, even if they are unlikely, they’d have to be really unlikely with a high degree of confidence for that to be sufficient grounds for not taking significant precautions.

This piece was first written on August 1, 2017, and first appeared on my website on June 11, 2025.