It’s easy to look around at the stupid seeming things that other people believe (e.g., people who join harmful cults, get scammed by a con artist, become vocal evangelists for a placebo treatment, or jump on the hype train of some outrageous new bubble) and wonder: “How on earth can they be so dumb?”

The answer, a lot of times, is simply the trust they have in someone else.
In other words, if a person were to evaluate the bad idea itself – call it X – they may well see it as dumb, dangerous, or full of hot air.

Instead, someone that person sees as impressive and totally trustworthy (or someone they just really like and respect) tells them that X is the next big thing. Or that X will change their life. Or that X will make them rich. Or that X will solve a problem for them that they desperately want solved.

This puts their brain in a predicament. They can either believe:
(1) That this impressive person who they deeply trust is deceiving them
Or
(2) That his impressive person who they trust is right – and their life will be way better off because of it!

If their trust in the person is great enough, or, at least, greater than their level of skepticism, (2) may win them over simply for that reason.

But (2) may also win them over for one or more of these reasons:

• they so desperately want this to be real – they so want to be special, or rich, or to have their biggest problems finally solved
• it’s difficult and painful to believe this person they trust so much is deceiving them or so wrong about something important
• they sense it will damage the relationship if they refuse to believe, and they care deeply about the relationship
• they have a hard time saying ‘no’ – perhaps it makes them very anxious to do so

In other words, there are a great many dumb things that even smart people end up believing in simply because people believe people. To be clear, this is not the only mechanism by which smart people fall for dumb things. Being smart is not the same as acting rationally all the time. But this trust-based force is, I think, an important mechanism.

While a belief in others is wonderful and admirable in many instances, it can also be a chink in our skepticism and rationality. It can lead us to believe in crazy and dangerous things that we wouldn’t be likely to believe without that trust. We see this when people get scammed by their favorite influencer or when they become true believers in quack cures because they have a friend who says it changed their life.

While this effect often happens when one person we trust causes us to believe in X, the effect is magnified when more people around us believe. Being recruited into a harmful cult by a trusted friend can be difficult, but leaving a cult – at which point all of our close friends are believers in X – is far more difficult. And growing up in an authoritarian regime – where EVERYONE we’ve met seems to believe in X, makes X that much more impossible to resist.

When rationality is discussed, it’s often talked about at the level of the individual. But quite a bit of our thinking we necessarily outsource to others – we can’t make sense of everything ourselves. When we allow someone into our circle of trust who doesn’t deserve to be there, that can jeopardize our rationality. Hence, an important meta-skill of rationality is knowing who to trust – and not being suckered into trusting those who don’t deserve it.

Almost everyone is susceptible to this phenomenon of being duped because of our trust in people, but that doesn’t mean there’s nothing we can do to avoid it.

One thing that I think helps is to treat trust as being multi-factor. I can trust a person in one way or in one domain but not another. Or, put another way, earning trust is multi-dimensional. I can see someone as trustworthy because:

• I know they wouldn’t betray me and that they care a lot about me
• I know that they vet evidence carefully, come to their beliefs in a rigorous way, and approach new information skeptically
• I know that they are extremely knowledgeable about a specific topic area

Being strong in one of these domains doesn’t automatically make someone strong in another. So, viewing someone as trustworthy in one of these domains shouldn’t cause you to view them as trustworthy in the other ones. And yet, that’s what many people do.

If you track trust in a one-dimensional way, it puts you at a lot of risk because someone you trust may have a very bad idea that they really want you to believe in. It may be hard to reject that idea because you trust them so much – and that may mean joining a harmful cult, buying into the peak of the next bubble, putting stock in an ineffective treatment, or being scammed.

This piece was first written on September 13, 2024, and first appeared on my website on September 22, 2024.

How to do an Instantaneous Experiment:

Step 1: Think of anything at all about the world that’s checkable that is likely to be true if your hypothesis is true, but that is likely to be false if your hypothesis is false.

Important: this checkable thing should be something that you have never investigated before – in other words, you don’t actually know if it’s true, and the only real reason you think it’s true is just because your hypothesis implies it would be. This is critical to help prevent bias from occurring during the process (for instance, this procedure doesn’t work if the fact you are checking is one that influenced your development of the hypothesis).

Step 2: Go check whether the checkable thing is true or not by trying to look the answer up (e.g., in an article or paper)!

The amount of evidence that the answer provides in favor (or against) your hypothesis precisely depends on how many times more likely you are to see that result if your hypothesis is true compared to if it’s not true. The bigger that number is, the greater the evidence!

Instantaneous Experiments work because, to get evidence for a theory or hypothesis, it is not necessary to directly check whether that thing is true. All you have to do is check something that is implied by that theory (that would be unlikely to be true otherwise).

Here’s an example:

Suppose you believe that “greater intelligence causes people to worry a lot more”

That’s very hard to test. But you can do an Instantaneous Experiment:

Step 1: if intelligence causes worry, then you might expect higher IQ people to agree more often with a statement like “I worry too much,” whereas if the theory is not true, you wouldn’t expect a positive correlation between IQ and agreement with that statement.

Step 2: We go check this, and we find a paper that measures both IQ and the level of agreement on the statement “I worry too much.” The correlation between them is essentially 0.

Result: We haven’t completely disproven the theory, but we should now reduce our confidence in it compared to what we thought before.

How much we reduce our confidence depends on how many times less likely we’d be to find no correlation between self-reported worry and IQ if our hypothesis “greater intelligence causes people to worry a lot more” is true, compared to if it’s false.

This piece was first written on August 24, 2024, and first appeared on my website on October 11, 2024.

1. Soldier Altruists think it’s obvious how to improve the world and that we just need to execute those obvious steps. They see the barriers to a better world as:

(i) not enough people taking action (e.g., due to ignorance, selfishness, or propaganda), and

(ii) bad groups blocking things (e.g., corrupt politicians or greedy corporations).

2. Scout Altruists think it’s hard to figure out how to improve the world – and most attempts either don’t work, only slightly help, or make things worse. They see the barriers to a better world as:

(i) not enough understanding of causal mechanisms (e.g., due to a lack of high-quality evidence, not enough attention to the evidence we do have, not enough careful reasoning, etc.), and

(ii) too much investment in bad solutions (e.g., due to people jumping to conclusions, doing what feels good emotionally rather than what is effective, ideological blindspots, inertia, etc.)

Soldier Altruists say we need to DO and FIGHT more. Scout Altruists say we need to THINK and TEST more. Soldier Altruists are more likely to think that if we could just get people to be less selfish and more motivated to act, we would make a lot of progress towards a better world. Scout Altruists are more likely to think that if we could just get people to pay more attention to evidence and to have more good-faith debates with strong norms around the quality of argumentation, we would make a lot more progress.

Soldier Altruists may think Scout Altruists are far too reluctant to act and are wasting their time on research and debate. Scout Altruists may think Soldier Altruists are far too confident in their conclusions and are wasting their effort pushing for changes that aren’t going to help much (and which, in some cases, might even make things worse). Of course, in reality, there is a continuum between these two positions. So, on a scale from 0 (Soldier Altruist) to 10 (Scout Altruist) where do you fall? I’m probably a 7 or 8.

As some commenters have pointed out, there is a relationship between this distinction and “Conflict Theory” vs. “Mistake Theory.” I think it is related – but also distinct in important ways. Conflict theory says that there is a giant zero-sum struggle (groups fighting over fixed resources). Whereas in this case, we’re operating from a framework of altruism: “the world can be made a lot better – what’s the big barrier to that happening? Is it that we know what to do and we’re not doing it enough/with enough energy, or is it that we don’t really know what to do?”

Also, to clarify another important point brought up in the comments: I’m not asking, “do you think it’s obvious how we should improve the world if we had a magic wand that could change whatever we wanted?” – instead, the question is: “is it obvious what to do to improve the real world, given that we don’t have a magic wand?” Do we just need to put more money/time/effort/people into executing the current “obvious” strategies because they will work well if we just scale them up? Or is it pretty unclear what strategies we should even be putting more resources into (meaning that a lot of thinking, research, debate and/or evidence evaluation will typically be necessary to even figure out what is worth scaling up)?

Julia’s book (which I highly recommend): https://www.amazon.com/Scout-Mindset-Perils-Defensive-Thinking/dp/0735217556/ref=nodl_

This piece was first written on April 23rd, 2021, and first appeared on this site on January 7th, 2022.