These four categories of moral reactions to behavior are:

1) Disgust: A visceral, emotional reaction to a behavior (which appears connected to the moral realm) that’s perceived as gross or disgusting.

Examples:

• The disgust that most people would have at the idea of someone eating human flesh (even in a survival scenario where there is no other food available and a person has died of natural causes).

• The disgusted reaction that some (but not most) people have to the idea of gay sex. For instance, in one study I ran, about 40% of Americans said that imagining themselves having sex with someone of their own gender caused them “an emotion of disgust.”  

I think that people sometimes confuse the visceral emotion of disgust in response to a behavior with the behavior itself actually being immoral. Jonathan Haidt’s “moral dumbfounding” experiments support this point of view, where people insist a situation is immoral but can’t explain what is immoral about it (because the situations were carefully crafted not to violate moral principles and not to involve harm).

2) Character judgments: when a behavior is seen as indicative of an unethical character, even in cases where the behavior itself has no actual effects.

Examples:

• A person going to watch the daily operations of a slaughterhouse because they are intrigued by the idea of watching an animal die

• A person who enjoys daydreaming about stealing items from people they know, even though they have never stolen before

Cases like these provide evidence (some would argue, though it’s perhaps debatable) that a person has bad moral character, even if the behaviors themselves are not immoral. But people can jump to thinking an action is immoral because the sort of person that does it is more likely to be immoral, which I’d argue is, a mistake. While immoral actions are evidence of bad moral character, some actions that are evidence of bad moral character are not themselves immoral actions!

3) Heuristics: when a behavior is (collectively) judged as being “bad” because it often either causes harm, involves unethical behavior, or involves defecting on a social contract of some sort. These negative judgments can apply even if the behavior in question isn’t causing any harm in the specific situation where it’s being witnessed.

Examples:

• Driving fifty miles an hour over the speed limit is generally categorized as bad behavior, but if someone did it because they were driving a dying person to the hospital, we’d say that was actually okay.

• The head of an organization dating one of their employees is generally viewed as bad behavior because it often leads to harm, but there are instances where many people would say that in that particular case, it was ethically okay, such as cases where the employee insistently initiated the relationship leading to the pair ending up happily married

It’s easy to think of violations of heuristics like these as being bad, but really what’s going on is that we’re socially agreeing they are bad because it’s a good and helpful rule of thumb to treat them that way. But special circumstances can make them fine. Because of this, it’s not unreasonable to judge actions as probably bad when they fall in these categories (when we lack other information), but we should be sensitive to the specific details of the case since they are not necessarily bad.

4) Fundamentally unethical: a behavior that is, in and of itself, actually unethical according to at least some people’s deep ethical values. These actions don’t necessarily cause a feeling of disgust, and don’t merely appear bad because they are the sort of thing bad people do, and aren’t merely matching a heuristic about what’s bad – they are actually bad because of the precise action for some ethical values that people hold.

Examples:

• Poisoning your toddler because you don’t want a child

• Pretending you love someone when you don’t because you lack the courage to be honest

• Secretly spying on someone so you can see them naked

• Violating a promise you swore to uphold merely because you’re feeling lazy

When we treat an action as fundamentally unethical merely because it produces a feeling of disgust, or because it’s the sort of thing that provides evidence of bad character, or only because we have a societal heuristic against it because actions in that category tend to be harmful, I think we’re making a mistake. These categories are easy to conflate with an action being immoral, but they aren’t the same thing.

This piece was first written on November 16, 2024, and first appeared on my website on January 20, 2025.

Here are five ways causation without correlation can occur:

1. Averaging: increasing A sometimes causes increasing B, but other times, it causes B to decrease. The two balance out. Since correlation measures the average relationship, the correlation is zero.

For example, if you drive up a symmetrical hill and then down the other side, there’s no correlation between how many times the wheels have revolved on the hill and the car’s height above sea level, even though the revolving of the wheels causes the altitude to change.

2. Confounders: when A increases, that causes B to increase, but change is coming from C, and increasing C both causes A to increase AND causes B to decrease. The decrease in B from more C is the right amount to counteract the increase in B caused by C causing A to increase.

Example: A is in love with B and best friends with C. When C is happy, that makes A happy. When B compliments someone in front of A, that makes A unhappy. When B complements C, that makes C happy, making A happier, but the complement itself makes A unhappier, balancing it out.

3. Control: a system is designed to stabilize the value of B. Even though increases in A normally cause an increase in B, the system resists this change by exerting the opposite effect on B with the goal of keeping B constant.

For example, opening the vents would normally cool down the house, but when the heating system detects a draft, it causes the heating system to start working harder in order to keep the house at the desired 70 degrees.

4. Multiple causation: while A does cause C, C is also caused by B (so either A or B on its own is sufficient to cause C). Since A and B are both active, deactivating one has no effect on C. A and B never get deactivated at the same time, so the relationship between A and C is 0.

Example: on death row in a certain country, they administer a lethal dose of two different poisons. They have experimented with removing one or the other poison, and this did not affect whether the inmate died.

5. Deactivated causation: an increase in A causes an increase in B normally, but only if C is active. Since C is not active, there is no correlation measured between A and B. For example, force on the gas peddle only causes the car to increase speed when the car is turned on

All of this being said, while causation does not NECESSARILY imply correlation, causation USUALLY DOES imply correlation. Some software that attempts to discover causation in observational data even goes so far as to make this assumption of causation implying correlation.

This piece was first written on March 14, 2022, and first appeared on this site on December 7, 2023.

We often say things like “I believe…” and “they think that…”

But what do we really mean by a “belief”? It’s notoriously tricky to define.

For starters, we sometimes think of beliefs in binaries (true vs. false) and other times in probabilities (a 90% chance of coming true). We sometimes would be willing to bet on our beliefs (“I’ll bet you $100 that New York City is not the capital of New York State”), and other times we wouldn’t be willing to bet (e.g., that your favorite team will win the Super Bowl, even though you may feel confident about it). It seems, sometimes, like we don’t fully believe our beliefs (e.g., we say, “I know it’s not dangerous,” and mean it, but then act as though it’s dangerous).

So what’s going on here? My theory: what makes discussing beliefs so confusing is that there are actually many different mental states we can have that are “belief-like.” In other words, beliefs are not one type of thing. They come in many forms.

We usually lump these divergent forms together, which creates a lot of confusion. At best, we divide them into dichotomies like explicit, cognitive “beliefs” vs. automatic “aliefs,” which still combine disparate forms together.

Below is my ambitious attempt to distinguish all the different belief-like states our minds can have.

I’m sure I missed some. So, I’d be very interested to know: what types of beliefs am I missing?

FORMS OF BELIEF

Beliefs On Reflection

Endorsed Belief: there are some things we’re willing to sincerely say we believe (or willing to say we’re X% confident about), and this is a form of belief – the things we “believe” that we believe.

Ex: you write as a social media post, “I believe healthcare should be free for everyone,” and there is no doubt in your mind that you really do endorse this.

Simulation Belief: we can ask ourselves hypotheticals like: “if I suddenly took my clothes off in the street, how would people react?” Our brains will then simulate the scenario and provide the best guess of what will happen. This is a form of belief about what would happen.

Ex: you consider how your friend would react if you told them you don’t like their haircut, and your belief is that they would be angry at you.

Automatic Beliefs

Anticipation Belief: our brains constantly predict what is about to happen (based on current sensory input, knowledge, and what happened recently). When these anticipations are far off from what actually happens, we feel surprised. These are a kind of “belief” about what is about to happen in the next moment.

Ex: you knock over a full cup of coffee and anticipate it will spill onto the ground. If it doesn’t spill, you will be surprised.

Sensed Belief: when someone makes a claim, such as “most pirates are ninjas,” we have a felt sense of whether we believe that statement or not (and how strongly/confidently we believe it). This is just a feeling, but it is a feeling about our level of “belief.”

Ex: you say to yourself, “I believe I am a good person,” and then you consider that statement carefully to see if you really FEEL like you believe it.

This idea relates to techniques like Focusing, where you learn to pay attention to the “felt sense” of whether a statement resonates with you.

Emotional Belief: our emotions activate in specific sorts of situations (e.g., risk -> anxiety, contamination -> disgust). So if our emotions are activated, they can be interpreted as a form of “Emotional Belief.” If I’m anxious about X, on some level, I believe X could go badly.

Ex: you know it’s totally safe to walk across the balance beam suspended above the pit of foam cubes, but your heart is pounding in your chest, and your sympathetic nervous system seems convinced you’re about to plummet to your death.

Intuitive Moral Belief: we have feelings about whether most things are good, neutral, or bad (and to what degree). For instance, you might like tea and hate coffee. You might like Biden and dislike Trump. These are beliefs of a sort – about what’s good.

Ex: are guns good or bad? What about nuclear power? Youth? Avocados? Spiders? If you pay close attention, you’ll probably realize you have an automatic sense of how good or bad these things are.

Self-serving Belief: sometimes, we want something to be true badly, and we won’t even allow the possibility that it’s false (e.g., because the thought that it could be false gives us pain, and so we immediately flinch away from that thought).

Ex: Some people might say, “my partner has never cheated on me,” without actually even considering whether it could be true.

Association Belief: we associate ideas with each other. For instance, we might associate milk with health (because of those “got milk” commercials) and Segways with nerds. These are a sort of implicit belief about the nature of these things (health effects / who uses them). (These beliefs can also be thought of as automatic or implicit memory-based beliefs.)

Ex: do you associate cities with smog? Do you associate French people with an enjoyment of food?

Implied Belief: you have probably never considered whether 384883828382553 is a number, but you already believed it is, in the sense that you have beliefs about what makes something a number. By implication, they imply it is one. Beliefs can imply other beliefs.

Ex: if you believe all men are mortal, and you believe Elon Musk is a man, you also, in a sense, believe that Elon Musk is mortal, even if you’ve never thought about the mortality of Elon Musk.

Memory-based Beliefs

Autobiographical Belief: suppose someone asks you if you have ever eaten Flaming Hot Cheetos. If you can recall an instance of doing so, you’ll say you have eaten them; otherwise, you won’t. One type of belief is what we can recall being true. Another way this can manifest is if we’re considering a question (like: “Is Sally a flaky friend?”) we may try to recall an instance of her being flaky, and if we can, we conclude she is flaky, whereas if we can’t, we are more likely to conclude she is not.

Ex: have you called a friend on their cell phone in the past seven days? If you can recall a case of doing so, you’ll believe it’s true. If not, you’ll very likely believe it’s not.

Memorized Belief: you can believe a statement in the sense of memorizing that the statement is supposed to be “true.” For instance, if you’re taught from age three that “Morloc is the sky god,” You say “Morloc is the sky god” and believe you believe it. You may not know what it means.

Ex: TV rots your brain. Opposites attract. Do you believe these statements are “true” because you heard that they were before you’ve had a chance to reflect on whether or not they really are true?

Elicited Beliefs

Generated Belief: when we are asked a question (by ourselves or others), we usually will generate an answer (e.g., “Why did you do that?” -> “Because I was angry”). This answer is a kind of belief related to the query. On reflection, though, we may decide we don’t believe it.

Ex: your friend asks you whether you want to go camping. You immediately blurt out “yes” – but then you immediately start considering whether or not you would actually enjoy camping.

Queried Belief: suppose you are shown a picture of a person and asked to “predict where this person is from.” A guess will likely appear in your head. In a sense, you have a belief that this person is from this place (arguably even if you have not yet run that mental query).

Ex: What country in the world do you think has the smallest landmass? Your brain may well generate a guess upon hearing that query.

Reactive Belief: we may find that a thought pops into our head (e.g., “Joe is a jerk”), and some people may view merely having this thought as a form of belief. Of course, upon reflection, we may or may not agree with the thought.

Ex: upon seeing Marty throw the bowling ball into the next lane by accident, you may have the thought, “Marty is awful at bowling.”

Behavioral Beliefs

Enacted Belief: sometimes, we believe something in order to create a state of affairs that is true by virtue of having the belief. For instance, we may think that “good people believe X” and we find that we can get ourselves to believe X, so we try to believe it, or we may think “my company will succeed if I believe it strongly enough” and so we push ourselves to believe we will succeed in hopes of making it true (e.g., dismissing any thoughts that we will fail, and trying to focus just on evidence in support of our future success).

Ex: someone who believes deeply in the placebo effect might do everything they can to believe the medicine will make them feel better, with the theory that, if they believe hard enough, it will work – which means then that they will be justified in that belief. (H/T Alli Smith and Annie Kotowicz.)

As-if Belief: sometimes we believe something in the sense of simply acting as though it is true, whether or not we would say we are highly confident in it. For instance, we might have heard a rumor that our neighbor once went to prison for molesting a child. We are considering how to act towards this neighbor. We may conclude that, even though there is uncertainty (since it’s just a rumor), we will act “as if” it is true – even though, if pressed on the topic, we will say we are uncertain whether it is true. Or we could decide the opposite (to act as though it is NOT true, on the principle that we shouldn’t act as if someone committed a criminal act unless we have strong evidence – like the principle of being “innocent until proven guilty”). 

Sometimes these “as-if” beliefs are much deeper; for instance, we might act “as-if” induction works, even if we know we can’t provide strong arguments in favor of it, or we might act “as-if” god doesn’t exist, even if we aren’t really sure about it. Or we might act “as-if” we have a death wish (e.g., by engaging in extremely risky behavior), even though we don’t think we have any desire for self-harm. H/T to Pepe Le Pew for inspiring this belief type.

There are many forms of belief. In plenty of cases, it’s unnecessary to differentiate between them, but on complex questions of human psychology, we may need to get granular with the idea of “belief” to really understand what’s happening.

Consider advanced cases like these, that are hard to make sense of without a nuanced perspective of the different forms of belief:

• Why do people who are absolutely convinced they’re going to heaven get scared of dying?

• Why do people claim that Trump or Biden is definitely going to win but then refuse to make a bet on that claim (even though normally they enjoy betting)?

• Why do we say things like “I believe in love,” even though, when asked what it means, we may struggle to explain it, and we may not have thought in detail about its meaning before?

This essay was first written on October 11th, 2020, and first appeared on this site on January 14th, 2022.

The idea that you should “just trust your gut” – that is, make many life decisions solely based on intuition (as opposed to based on reflection) – is obviously very popular. But I think that there are pretty much only four types of situations where we’re best off relying on intuition alone: when a decision is Fast, Irrelevant, Repetitious, or Evolutionary (FIRE for short).

Case 1: Fast decisions
There is no choice but to make the decision quickly, so thinking it through is infeasible. In this case, intuition is your only option because it’s the only method of deciding that’s fast enough.

Examples:
– the car barreling towards you in the other lane has just swerved into your lane
– you’re in a job interview and are asked whether you’d still be interested in this job if it pays less than your prior job
– the train you’re thinking of taking is about to depart, and there is no other train for five hours

Case 2: Irrelevant decisions
The decision is of low importance, so minimizing time, effort, and indecisiveness is more important than optimizing for the best outcome. In this case, intuition is your best option because it’s the least costly.

Examples:
– you are trying to decide whether to put carrots on your salad
– you are trying to decide what TV show to watch tonight
– you are trying to decide which $5 product to buy among several similar options

Case 3: Repetitious decisions
You have lots of experience making decisions in that realm AND received reliable information on how the decisions turned out. In this case, intuition is trustworthy because it’s been honed through practice with feedback.

Examples:
– a heart surgeon who is conducting her 500th heart surgery (but not a heart surgeon who is conducting her 2nd heart surgery)
– a digital marketer writing email newsletter headlines who has been tracking the performance of each such email for five years (but not one who doesn’t track the performance of the emails)
– a chess player making a decision about which move to make after playing chess daily for years (but not a chess player playing backgammon)

Case 4: Evolutionary decisions

It’s the sort of survival-relevant decision that our ancestors had to make regularly 20 thousand to 200 thousand years ago, and hence we should expect that evolution built us to have reasonable instincts in this domain. In this case, our intuitions are quite reliable: our genes endow us with these intuitions precisely because they helped earlier copies of our genes survive.

Example:
– should you eat that foul-smelling old food?
– is that person who has been staring at you likely to have malicious intent?
– should you walk on that injured knee that’s causing searing pain?

So, to recap, I think pure intuition is the right solution pretty much only for FIRE decisions, that is, those that are:

(1) Fast (because you don’t have time for anything else)

(2) Irrelevant (because the costs of other approaches are too high)

(3) Repetitious (because with lots of practice, when we’re receiving reliable feedback, our intuition becomes well-honed)

(4) Evolutionary (because certain types of decision-related instincts are built into our biology)

In other words, if you weren’t born with the ability to make the decision intuitively (non-evolutionary decisions), you didn’t learn to make that kind of decision intuitively (non-repetitious decisions), the stakes are high enough that it’s worth using the best approach (non-irrelevant decisions) and you have time to use another approach (non-fast decisions) then you should indeed use another approach.

So if a decision is not FIRE, you’re probably best off thinking the decision through carefully, discussing the decision with others, writing out pros and cons, etc. Intuition still plays a very important role in those cases, but it plays a supporting role (e.g., to help you figure out things like what you value, to help you estimate likelihoods, to help you synthesize lots of information into an overall judgment, etc.) rather than playing the only role.

I’ll emphasize since it is so important: intuition almost always plays a critical role in decision-making. The question here is just whether you’re merely (1) “going with your gut” (for the whole decision), or whether you are (2) feeding your intuitions (which might include intuitions about what you value, what you predict is true, what you feel, etc.) into a broader decision-making process.

I think that using just your intuition on important non-FIRE decisions tends to produce bad life outcomes.

So don’t just go with your gut. Go with your gut when your decision is FIRE, and otherwise, let your gut be a really useful tool, rather than letting it be in charge of the whole process.

Acknowledgments
Thanks to Sam Rosen for his help improving this framework. 

This essay was first written on April 7, 2019, and was first released on this site on August 30, 2021.

You can make inferences using…

(1) Deduction:

As a consequence of the definition of X and Y, if X then Y.

X applies to this case. Therefore Y.

“Plato is a man, and all men are mortal; therefore Plato is mortal.”

“For any number that is an integer, there exists another integer greater than that number. 1,000,000 is an integer. So there exists an integer greater than 1,000,000.”

Especially popular among philosophers and mathematicians?

Flaws: to apply to the world, you need to add in assumptions about the world, or to apply other methods of inference on top.

(2) Frequencies:

In the past, 95% of the time that X occurred, Y occurred.

X occurred. Therefore Y (with high probability).

“95% of the time when we saw a transaction identical to this one, it was fraudulent. So this transaction is fraudulent.”

Especially popular among applied statisticians and data scientists?

Flaws: You need to have a moderately large number of examples like the current one to perform calculations on, and the method assumes that those past examples were drawn from a process that is (statistically) just like the one that generated this latest example. Moreover, sometimes it is unclear what it means for “X” to have occurred. What if it’s something that’s very similar to but not quite like X that occurred – should that be counted? If we broaden our class of what counts or change to another class that still encompasses all of our prior examples, we’ll potentially get a different answer. Though, fortunately, there are plenty of cases where the class to use is fairly obvious.

(3) Models:

Given our probabilistic model of this thing, when X occurs, the probability of Y is 0.95.

X occurred. Therefore Y (with high probability).

“Given our multivariate Gaussian model of loan prices, when this loan defaults, there is a 0.95 probability of this other loan defaulting.”

Especially popular among financial engineers and risk modelers?

Flaws: hinges on the appropriateness of the parameterized probabilistic model chosen, may require a moderately large amount of past data to estimate free model parameters, and may go haywire if modeling assumptions are suddenly violated.

(4) Regression:

In prior data, as X and Z increased, the likelihood of Y increased.

X and Z are at high levels. Therefore Y.

“Height for children can be approximately predicted as an (increasing/positive) linear function of age and weight. This child is older and heavier than the others, so we predict he is also taller than the others.”

Especially common among economists and data scientists?

Flaws: often is applied with simple assumptions (e.g., linearity) that may not capture the complexity of the inference, but very large amounts of data may be needed to apply much more complex models (e.g., to use neural networks).

(5) Bayesianism:

Given my prior odds on Y being true…

And given evidence X…

And given my Bayes factor, which is my estimate of how much more likely X is to occur if Y is true than if Y is not true…

I calculate that Y is far more likely to be true than to not be true (by multiplying the prior odds by the Bayes factor to get the posterior odds).

Therefore Y (with high probability).

“My prior odds that my boss is angry at me were 1 to 4, because he’s angry at me about 20% of the time. But then he came into my office shouting and flipped over my desk, which I estimate is 200 times more likely to occur if he’s angry at me compared to if he’s not. So now the odds of him being angry at me are 200 * (1/4) = 50 to 1 in favor of him being angry.”

Not as popular as it should be?

Flaws: it is sometimes hard to know what to set your prior odds at, and it can be very hard in some cases to perform the calculation. In practice, carrying out the calculation might end up relying on subjective estimates of the odds, which can be especially tricky to guess when the evidence is not binary (i.e., not of the form “happened” vs. “didn’t happened”), or if you have lots of different pieces of evidence that are partially correlated. On the other hand, if you can do the calculations in a given instance, and have a sensible way to set a prior, this is, in my opinion, the mathematically optimal framework to use for probabilistic prediction. In that sense, we can think of many of the other approaches on this list as (hopefully pragmatic) approximations of Bayesianism (sometimes good approximations, sometimes bad ones).

(6) Theories:

Given our theory, when X occurs, Y occurs.

X occurred. Therefore Y.

“One theory is that depressed people are most at risk for suicide when they are beginning to come out of a really bad depressive episode. So as depression is remitting, patients should be carefully screened for potentially increasing risk factors.”

“When inflation rises, unemployment falls. Inflation is rising, so unemployment will fall.”

Especially popular among psychologists and economists?

Flaws: it’s very challenging to come up with reliable theories, and often you will not know how accurate such a theory is. Even if it has substantial truth to it and is often right, there may be cases where the opposite of what was predicted actually happens, and for reasons that the theory can’t explain.

(7) Causes:

We know that X causes Y to occur.

X occurred. Therefore Y.

“Rusting of gears causes increased friction, leading to greater wearing down. In this case, the gears were heavily rusted, so we expect to find a lot of wearing down.”

“This gene produces this phenotype, and we see that this gene is present, so we expect to see the phenotype.”

Especially popular among engineers and biologists?

Flaws: it’s often extremely hard to figure out causality in a highly complex system, especially in “softer” subjects like nutrition.

(8) Experts:

This expert (or prediction market, or prediction algorithm) X is 90% accurate at predicting things in this general domain of prediction.

X predicts Y. Therefore Y (with high probability).

“This prediction market has been right 90% of the time when predicting recent baseball outcomes, and in this case, they predict that the Yankees will win.”

Not as popular as it should be?

Flaws: you often don’t have access to the predictions of experts (or of prediction markets or prediction algorithms), and when you do, you usually don’t have reliable measures of their past accuracy.

(9) Metaphors:

X, which is what we are dealing with now, is metaphorically a Z.

For Z, when W is true, then obviously Y.

Now W (or its metaphorical equivalent) is true for X. Therefore Y.

“Your life is but a boat, and you are riding on the waves of your experiences. When a raging storm hits, a boat can’t be under full sail. It can’t continue at its maximum speed. You are experiencing a storm now, and so you too must learn to slow down.”

Especially popular among self-help gurus and some ancient philosophers?

Flaws: Z working as a metaphor for X doesn’t mean that all (or even most) solutions that are good for situations involving Z are appropriate (or even make any sense) for X.

(10) Similarities:

X occurred, and X is very similar to Z in properties A, B, and C.

When things similar to Z in properties A, B, and C occur, Y usually occurs.

Therefore Y (with high probability).

“This conflict is similar to the Gulf war in that…and with “Gulf”-like wars, we have always seen that…”

“This data point (with unknown label) is closest in feature space to this other data point which is labeled ‘cat,’ and all the other labeled points around that point are also labeled ‘cat,’ so this unlabeled point should also likely get the label ‘cat.’”

Especially popular among historians and within some machine learning algorithms?

Flaws: in the history case, it is difficult to know which features are the appropriate ones to use to compare similarities, and often the conclusions are based on a relatively small number of examples. In the machine learning case, a very large amount of data may be needed to train the model.

(11) Cases:

In this handful of examples (or perhaps even just one example) where X occurred, Y occurred.

X occurred. Therefore Y.

“The last time we elected a [insert political group you don’t like] as president, we saw what happened. Let’s not make that mistake again.”

“The last three times I went to action movies, I didn’t like them. So I don’t want to go to one again.”

Especially popular with politicians and with nearly everyone in daily living?

Flaws: unless we are in a situation with very little noise/variability, a few examples likely will not be enough to accurately generalize from.

(12) Intuition:

X occurred. My intuition (that I may have trouble explaining) predicts that when X occurs, Y is true. Therefore Y.

“The tone of voice he used when he talked about his family gave me a bad vibe. My feeling is that anyone who talks about their family with that tone of voice probably does not really love them.”

Popular with nearly everyone in daily living?

Flaws: our intuitions can be very well-honed in situations we’ve encountered many times and that we received feedback on (i.e., where there was some sort of answer we got about how well our intuition performed), but in highly novel situations or in situations where we receive no feedback on how well our intuition is performing, our intuitions may be highly inaccurate (even though we may not FEEL any less confident about our correctness).

This essay was first written on October 7, 2018, and first appeared on this site on December 31, 2021. 

Here are the materials, in case you want to run an event like this or just learn about the format:

Part 1: Brief presentation about why disagreements are so often unproductive

1. You want to “win” more than you want to figure out what’s true (e.g., seeing disagreement as a competition)

Example: you want to one-up the other person, the other person is a rival

2. It’s really about your group identity, not about the content of the disagreement per se (i.e., what the argument is really about is whether my group is better than your group)

Example: Republicans vs. Democrats, Yankees vs. Mets

3. Others are watching and you think backing down would lower your social status or lose you respect (e.g., you think it would look bad to back down)

Example: dinner party disagreements, public debates

4. You have emotional discomfort with finding out you are wrong on this topic (e.g., cognitive dissonance, feeling dumb for being wrong, realizing that you did something immoral in the past)

Example: you’ve already said you are “certain” you are right, so think you would look really dumb now being wrong

5. There would be real-world negative consequences to believing the truth about this, which your subconscious detects, and so you fight against learning the truth (e.g., social rejection, feeling that you’d need to abandon your current lifestyle)

Example: religion, eating animals, addiction

6. You feel pressure to back down in the disagreement or that it would be inappropriate to mention flaws in the other person’s argument [H/T: Divia]

Example: disagreement with your boss, or in an especially polite social context, or with a person who you think would react negatively to be being challenged

7. The conversation drifts, so at first it was about your disagreement over X, but now it is about your disagreement over Y, so you can’t make progress on X

Example: pointing out whenever the other person says something you disagree with, even if it is tangential to the main point

Part 2: Tips for having productive disagreements that both parties learn more from (for attendees to keep in mind during the interactive portion of the event):

Tip 1: Remember that you are here to learn from the other person, not to “win” an argument

Tip 2: Work from the assumption that the other person has good intentions and is not a bad person, even if you think they believe something harmful

Tip 3: If the other person changes your mind about anything (even something small), tell them so and thank them for helping you have a more accurate perspective, changing your mind means you’re doing this right!

Tip 4: Carefully clarify definitions, since words are ambiguous and people often mean different things by the same word (once you do this you may realize that you don’t actually disagree at all)

Tip 5: Try to discover WHY the other person believes what they do, not merely just WHAT they believe

Tip 6: Remember that some of your beliefs are inevitably going to be wrong, it’s better for you to find that out now and start believing the truth as soon as possible rather than to hold tight to those wrong beliefs indefinitely

Tip 7: Remember that even if the other person IS wrong, you can still gain a more accurate view on the issue by understanding any valid points they are making

Tip 8: Remember that most controversial topics have at least some reasonable points on both sides, and you should know the best of what the other side can offer

Tip 9: Remember that even if your goal in the end is really just to persuade the other person, it still helps to ask lots of questions so that you deeply understand why they believe what they believe before trying to persuade them

Tip 10: Remember that we shouldn’t be 100% confident in any of our beliefs, so it’s better to think of your beliefs in terms of confidence levels (“I’m reasonably confident in this belief”) rather than in terms of things being either all true or all false, and talking to someone who disagrees with you can help you make your confidence level more appropriate even if you don’t end up concluding they are correct (e.g., after the discussion you might change your mind from being “very” confident to only being “moderately” confident)

Tip 11: Stay on topic, and hone in on the core of the disagreement, don’t get distracted by side disagreements or let the conversation drift away from the most important parts or you will inevitably not make progress, and if the other person says something you disagree with that is not that relevant to the topic, just let it go

Part 3: The interactive portion of the event

Handout 1

A spreadsheet of 59 controversial topics (print one out for each attendee): everyone will fill this out. If a person doesn’t want to discuss a certain topic, they should mark it as “unsure” or just leave it blank.

Important note: while this might sound like a huge number of controversial topics, it seems that a large number of topics is actually needed since many items people will not have an opinion on, or will agree with most other group members on (due to meeting groups often having similar views in many domains), or will find too ambiguous. If you write your own controversial topics, I don’t recommend using less than 50.

Here is another spreadsheet of 59 controversial topics adapted for people in the effective altruism community.

Handout 2

Step by step instructions for the interactive portion of the event (print one out for each attendee):

Step 1.

Find a random person to be your partner, preferably someone you don’t already know.

Step 2.

Compare your sheet with your partner (overlaying them after folding the right edge of one sheet works well) to find a topic where one of you selected “agree” and the other selected “disagree.”

Don’t select a topic that either of you marked as “unsure” or that either of you left blank; those topics are off-limits.

Step 3.

(i) Pick who will start as Person 1 (the explainer) and who will start as Person 2 (the asker).

(ii) Person 1 then explains why they marked that they agree or disagree.

(iii) During this time, Person 2 asks clarifying questions, with Person 2’s only goal being to try to TRULY understand Person 1’s perspective.

Good questions for Person 2 to ask include, among other things:

A. What do you mean by the word X? (i.e., clarifying a definition)

B. Could you give me an example of that?

C. Could you explain why you think that’s true?

(iv) Person 2 then tries to repeat back to Person 1 the reasons that Person 1 said they agree or disagree with the original statement, until they can do so in a way that Person 1 confirms is an accurate characterization of Person 1’s view. Being able to state someone’s perspective back to them is a great way to confirm you’ve really understood it.

Step 4.

Now switch who is Person 1 and who is Person 2 and repeat exactly what you did in Step 3.

Step 5.

Spend a final few minutes discussing the topic with the goal of pinpointing exactly why you disagree.

(i) Is it just a semantic disagreement, where once you agree on the meaning of words and definitions you actually agree with each other? This happens surprisingly often!

(2) Is it a disagreement of values (e.g., one of you simply values autonomy more than the other)?

(ii) Is it a disagreement about empirical facts that one could check (e.g., there might be studies out there that would settle the disagreement)?

(iii) Or is it a disagreement about intuition (e.g., you have different intuitions about a topic but can’t really explain why)?

Your goal should be to discover the “core” of the disagreement, the ultimate explanation for why you have different perspectives.

Step 6.

Now you’ll rotate to another person and repeat the whole process over again.

This piece was first written on June 25, 2017. Grammatical edits were made and an EA-adapted version of Handout 1 was shared on June 27, 2018. This editing note was added on December 23, 2022.

Our non-conscious systems primarily work using pattern recognition. Through a combination of genetic pre-programming and repeated exposure, your brain learns to label instances of things in the world as “dangerous” or “not dangerous”, “food” or “not food”, “person” or “not person”, “real smile” or “fake smile”. It learns to categorize and predict automatically.

When you see a chair you are immediately able to recognize it as a chair and associate it with your concept of chair, even if it is the largest chair you have ever seen, or the first one you’ve seen with a zebra stripe pattern. Your brain has effectively developed a way to rapidly recognize things as being “chairs” or “not chairs” based on whether they match a certain pattern. This pattern was learned automatically from exposure to many chairs in the past. You don’t have to consciously consider whether this particular object has the features necessary to make it a chair, your brain produces an answer before you are even aware of thinking about it.

To give another example, consider what happens when a chess Grandmasters looks at a chess board. In many cases, they are able to glance at a board positions and immediately (without conscious thought) identify them as being strong or weak, with good reliability. Of course, they might then choose to reason consciously about the positions in order to check or improve upon their gut responses.

Having considered these two ways of making predictions and decisions, the question then arises: when should we rely on thorough conscious analysis, and when should we use our automatic processing? To answer this, let’s consider the advantages and disadvantages of these two approaches.

Careful Conscious Analysis

Advantages

• Handles problems where multiple logical deductive steps are necessary to find a solution (e.g. I know that all A’s are B’s, and also that if something is a C it cannot be a B, therefore this particular A is not a C.)
• Allows the application of theories to problems, and in doing so lets you leverage the research and thinking of others (e.g. what does economic theory tell us will happen to soybean prices when the supply of soybeans dries up?)
• Allows you to develop theories of your own which can then be applied in future situations. (e.g. I’ve noticed that when I attack him on his left side he parries and counter attacks. So this time I will initiate a fast attack to the left, and as he begins to parry, I will drop the point of my blade to get underneath his sword, and carry the momentum into an attack to the right instead)
• Lets you compare the advantages and disadvantages of multiple possible methods or solutions. (e.g. The first theory predicts results will come out a certain way, but the second predicts a slightly different outcome. However, the first theory has a better track record on cases like this one.)
• Produces reasoning that can be communicated to others. (e.g. I came to my conclusion by evaluating the randomized controlled trials and noting that…)

Disadvantages

• Unless you have honed your truth discernment skills, you may fall prey to logical fallacies, become misled by cognitive biases, or rely on theories that have not been empirically validated. Performing truly excellent and reliable conscious analysis takes work, skill, self correction, good habits of mind, knowledge about potential pitfalls, and practice. (e.g. Due to the compatibility of your astrological charts, I think that he would make an excellent husband.)
• Thinking carefully is a slow process. (e.g. There is a deer in the road about thirty feet ahead of me and I’m going 60 miles per hour. If I hit the brake now will I have time to stop before I hit it? Will it be more or less dangerous to me if…)

Unconscious Automatic Processing

Advantages

• Operates fast (e.g. That ball he hit is going out-of-bounds so there is no need to run and try to catch it)
• Can be used in lots of situations where feedback is readily available, even when you don’t have a theory about how to solve the problem consciously (e.g. Now when I look at a cubist work I immediately know if is by Picasso, Braque or someone else, even if I’ve never seen it before.)
• Starts to get trained automatically without effort if you just do something enough times (e.g. Having listened to enough songs that I knew the titles of, I can now often guess what the title of a song is just from its lyrics)

Disadvantages

• Cannot effectively handle all types of problems. This method is especially bad for problems that require long deductive chains to solve, or where gathering various sorts of evidence together is necessary. (e.g. I know that the randomized controlled trials say otherwise, but my own experiences as a psychologist still tells me that psychodynamic therapy is more useful and cost-effective for treating depression than Cognitive Behavioral Therapy)
• May give you wrong answers when the problem changes compared to what you have been trained on (e.g. I know we’re playing squash now, but that move I did would have been perfect if we were playing tennis!)
• Requires repeated exposure, so won’t work well when you don’t get to practice a lot or see examples over and over again. (e.g. Well, I’ve only done heart surgery two times before, but from what I can tell this doesn’t quite seem to be in the right place, so I’m going to abandon the standard procedure and improvise)
• Doesn’t always provide us with conscious insight into why the prediction came out a certain way (e.g. I know that thing is designed to put food onto, but it still looks like a chair to me)

Careful analytical thought, and automatic processing relying on pattern matching are both extremely useful ways of approaching problems. When processing needs to be done quickly and you can train yourself through lots of examples with feedback, automatic processing may provide the best method for making predictions. But when chains of logical deduction are required, or evidence needs to be gathered and evaluated, or repeated exposure is not possible, or strong predictive theories exist, then as long as you have honed your thinking skills sufficiently well, conscious analysis will probably provide the best results.

It is common to hear advice that amounts to “going with your gut,” “trusting your instincts,” or “following your intuition.” But these suggestions seem to indicate that the answers we arrive at through careful thought are sometimes (or perhaps usually) inferior to our immediate reactions. 

Consider the following representative quotes:

Keep the faith, don’t lose your perseverance, and always trust your gut extinct. – Paula Abdul, Performer

I always trust my gut reaction; it’s always right. – Kiana Tom, Television Host

Intuition comes very close to clairvoyance; it appears to be the extrasensory perception of reality. – Alexis Carrel, Nobel Prize Winner (Medicine)

Before making a decision about whether to trust our gut, it seems like a worthwhile endeavor to first try to understand what exactly our “gut” is. 

There are a number of (non-mutually-exclusive) possibilities:

1. Sometimes, gut feelings can be explained by our evolved instincts that are encoded in our DNA, which exist because they helped our ancestors survive in the world as it was hundreds of thousands of years ago. For example, we are likely to feel anxious when confronted with something unfamiliar (a large unknown animal, for example) or feel fear when we hear a loud noise. These sorts of reactions are very useful at times but typically only work well in scenarios involving danger or conditions that are similar to those that our prehistoric ancestors encountered. At a birthday party where balloons are being popped, a fear of loud noises is not useful. On the other hand, when walking in a canyon, a loud noise might indicate a cascade of rocks, and a fear response might save your life.

2. In many cases, our intuition is associated with our immediate emotional responses to stimuli. For example, we might get a positive feeling towards someone who has a warm smile or feel negatively towards a person who is raggedly dressed. 

While our emotional responses clearly are important (e.g., it’s important to feel sympathy towards a friend when they look sad and to feel fear when encountering someone who has hurt you in the past), they can also be distorted and counter-productive. For example, many people feel slighted if a friend fails to return their phone call. However, one is likely to feel better about the situation (despite his initial emotional response) if he reminds himself that there are all sorts of reasons that a friend doesn’t return calls, including being temporarily busy or absent-minded. On the other hand, those who trust in their initial response of hurt and anger (which might stem, for example, from a lack of confidence or insecurity) are more likely to blow up at the friend or act coldly during the next encounter, which could damage a generally fulfilling relationship. 

Unfortunately, it can be tricky to differentiate between emotional responses that are useful and improve our lives and those that are based on misperceptions and are unhelpful. Both of these types of emotional responses feel equally real when you are experiencing them, and it is often only when we carefully scrutinize them from a neutral point of view that we can tell the difference.

3. Sometimes, our “intuition” is based on repeated experiences that we access automatically. For example, a surgeon might get an uneasy feeling during an operation, which could be triggered automatically when she notices similarities between the current patient and difficult surgical cases she has had in the past. Another example would be a painter who suddenly has an impulse to add more thinner to the paint he is using, which could be a subconscious reaction based on years of art-making experience. 

Generally, when we repeat similar tasks enough times, their execution becomes automatic and begins to feel like an instinct. We might then say that we have developed an intuition for effectively completing the tasks. These kinds of responses can occur much faster than those controlled by conscious thought and thus can be very efficient and useful. Of course, if the initial learning phase that leads to one’s intuitions is not of good quality (e.g., we complete the task poorly while learning), the intuition will be of poor quality as well. 

Sometimes intuitions of this form are experienced in emotional form as an uneasy feeling, in particular when our (possibly subconscious) expectations (based on experience, training, or practice) do not match what we see. A day trader, for example, might decide to sell a stock due to an uncomfortable feeling about it. In reality, what may have happened is that the stock chart had a pattern that did not conform to his expectations. This intuition can be quite useful since those cases that don’t match our predictions can be especially difficult, require special attention, and present unknown hazards.

4. Societal conditioning can create a kind of gut reaction that operates without conscious awareness. If we are told enough times when we are young not to trust strangers, then we may feel immediately distrustful when a stranger strikes up a conversation. The trouble with responses of this kind is that they are often formed without thought or careful reflection. For example, if we grow up in an area where a lot of racism is present, we might be told that certain races are inherently dishonest and so could have strong intuitive reactions to people of those races based on this idea. Or if we are taught to believe that black cats bring bad luck, we might start feeling uneasy in a house with a black cat. Since the ideas that society conditions us with may not be true, it can be very valuable to be aware of these automatic reactions and to carefully evaluate their logical and empirical validity.

5. Our brains are capable of forming very strong associations (for instance, between the smell of a perfume and a particular person), and this can lead to what might be interpreted as “gut feelings.” For example, if we were attacked by dogs when we were young, we may feel anxious around dogs later in life, even when we know on an intellectual level that we are safe around them. Such associations clearly have a valuable survival purpose, but on the other hand, they can push us into self-defeating behavior (like refusing to visit the house of a friend because she has a dog). Our brain’s natural tendency is to be “better safe than sorry,” as your ancestors were very unlikely to die from forming spurious associations (such as that dogs are generally dangerous) but were at real risk of dying if they failed to form certain real associations (such as that tigers are dangerous).

As we can see, what people call their “gut” is most likely a hodgepodge of various brain activities, which include survival instincts, emotional responses, automatic recall of learned information, societal conditioning, and associations. 

It is not at all obvious that these systems will lead to good decision-making in general. Each has its very important uses, as well as significant domains where it will fail. A businessman cannot rely on fight or flight instincts when giving an important presentation, a person with an anxiety disorder cannot expect his emotional responses to be accurate representations of reality, a doctor cannot rely on his old impulses when learning a totally new surgical technique, and no-one can expect that their society will condition them only with true information. Hence, to make truly good decisions, it is essential to evaluate your intuition to see whether it is likely to be accurate in the situation at hand. Each time our gut tells us something, we cannot know whether to trust it until we understand where that feeling originates from.

Note (January 13, 2022): For more about when to trust your gut, watch my TEDx talk here.

If you liked this essay, you may also enjoy a later essay I wrote, which fleshes out the specific circumstances under which it might be advisable to trust your intuition: The FIRE Framework: deciding when to trust your gut

1. Murder violates an abstract principle that I would like all people to live by.

2. The Bible (or some other religious text) forbids murder.

3. As a result of evolution and natural selection, most people have an innate emotional “moral” aversion towards murder.

4. Murder is against the law.

5. Murder is labeled as “immoral” by most people in my society.

6. Murder usually reduces the total net happiness of society.

7. The idea of murder provokes in me a morality-related emotional state of “wrongness”.

8. Nearly all philosophies forbid murder or say it should be avoided.

9. Nearly all societies have laws that punish murderers or have customs that ostracize them.

10. Most people would feel a sense of guilt if they committed murder.

Unfortunately, dictionaries cannot clarify for us what the word “moral” means. For instance, one dictionary defines “moral” as “conforming to a standard of right behavior”. Looking up the relevant definition of “right”, we find “being in accordance with what is just, good, or proper”. But the definition given for “good” is just as vague and circular as were the definitions for “moral” and “right”. Checking another definition, the result is no better. It defines “moral” as “conforming to accepted standards of behavior.” Accepted by whom, and for what reason? The dictionary does not answer these questions, and hence does not provide us with an unambiguous explanation of what “moral” means.

Many well-respected philosophers begin by assuming that morality is a single, well-defined thing and argue about what properties it must have. But if we haven’t defined morality, how can we derive its properties? Are we even talking about the same thing as each other – if we’re not, then of course we can’t agree on the properties of that thing. As the list of interpretations above shows, there are many very different things that we might reasonably call “morality”, including our genetic moral intuitions created by natural selection, the societal rules that are deeply ingrained in us, religious laws, and certain abstract concepts about how to treat each other.

Some people claim that whenever someone says that an action is “moral”, all that person is doing is expressing a feeling or emotion about that action. It’s easily falsified that this is ALL that talk of “morality” is doing – Christians, Kantians, and Utilitarians at least sometimes use the word “moral” to refer to actions that are compatible with biblical teachings, the categorical imperative, and the happiness principle, respectively. These individuals likely have an emotional feeling that their systems of ethics are worthwhile, but nonetheless, they often speak of morality in direct reference to their philosophical systems, independent of their personal feelings. Many people who speak about what is moral at least genuinely believe themselves to be expressing a fact.

Ultimately, before we can decide whether a statement like “murder is immoral” is true, we must first decide what we mean by “moral”. When we don’t know the definition of a word, it is difficult to have a meaningful discussion that relies on it. If we decide that morality is simply whatever the law says, or is determined by what the Bible says, or is a genetic characteristic of human beings, then the question of whether “murder is immoral” becomes an empirical and factual one. We need only check the laws for our country, or search through the Bible, or study human genetics and behavior in order to answer questions about what is moral.

In practice, though, typically when statements about morality are made, there is rarely any explicit or even implicit definition of morality being used. Your average person relies on an intuitive sense of what is right and wrong. This intuitive sense is influenced by many factors, including our genetics, the standards of the society that we live in, the religion that we practice, our personal experiences, and the philosophies that appeal to us. Unfortunately, it appears as though questions about morality are usually unanswerable without further specification about the sense in which “moral” is being used.

If the argument made thus far is true, then how can we understand the fact that nearly everyone seems to agree when it comes to certain ethical statements? For example, how can we account for the fact that almost all people in most of the societies that have ever existed have believed that most kinds of murder are immoral? Well, one relevant factor may be that a strong predisposition to viewing as bad the maurder of close others (especially the murder of family members) is inherent in the human genetic code. More generally, our sense of what is morally wrong appears to be strongly correlated with what we feel an emotional revulsion towards, and those things that we find repulsive are influenced by our genetics. If most humans share a “moral feeling” that is caused by the genes that we share in common, then that provides a plausible explanation of why, for example, murder is generally thought to be immoral.

It is not difficult to imagine that when pre-humans lived in groups, an aversion towards certain types of murder could increase an individual’s chance of survival (perhaps because would-be murderers had a high chance of being killed by their intended victim or the victim’s family). If this were the case, then the process of natural selection could help make a revulsion towards murder a common trait among our ancestors. It may be illuminating to note that many types of carnivores, though feeding daily on other (typically smaller) species, very rarely kill members of their own species (even during fights that break out). This is likely due, at least in part, to the fact that members of a single species are usually fairly evenly matched in strength and fighting skills.

A lion is very unlikely to be killed attempting to kill an antelope. Still, he is fairly likely to be killed or seriously injured when attempting to kill another lion, so lions that focus on eating antelope rather than killing other lions may tend to pass down their genes more effectively (even though there are obvious reasons why one lion might benefit if it does manage to kill another). What’s more, social species may ostracize the members of their group who they feel threatened by, which could dramatically reduce the chance of survival for a “murderer” (by which, in this context, I mean a creature that kills members of its own species). A “moral feeling” would be one possible way, among many, that our genes could urge us not to kill members of our own species.

Another reason for the high convergence in moral views about murder being bad may simply be due to the act itself – it is one that, on nearly any moral view, is bad. But different moral theories, while agreeing on its badness, disagree on why it’s bad and what exactly counts as murder. For instance, they may disagree about whether killing someone in self-defense is murder, or as part of war, or as a punishment for a serious crime.

It’s hard to make progress when discussing moral questions when people mean very different things by “moral,” which, very often, they do.

This piece was first written on July 22, 2009, and first appeared on my website on February 10, 2026.