It’s easy to look around at the stupid seeming things that other people believe (e.g., people who join harmful cults, get scammed by a con artist, become vocal evangelists for a placebo treatment, or jump on the hype train of some outrageous new bubble) and wonder: “How on earth can they be so dumb?”

The answer, a lot of times, is simply the trust they have in someone else.
In other words, if a person were to evaluate the bad idea itself – call it X – they may well see it as dumb, dangerous, or full of hot air.

Instead, someone that person sees as impressive and totally trustworthy (or someone they just really like and respect) tells them that X is the next big thing. Or that X will change their life. Or that X will make them rich. Or that X will solve a problem for them that they desperately want solved.

This puts their brain in a predicament. They can either believe:
(1) That this impressive person who they deeply trust is deceiving them
Or
(2) That his impressive person who they trust is right – and their life will be way better off because of it!

If their trust in the person is great enough, or, at least, greater than their level of skepticism, (2) may win them over simply for that reason.

But (2) may also win them over for one or more of these reasons:

• they so desperately want this to be real – they so want to be special, or rich, or to have their biggest problems finally solved
• it’s difficult and painful to believe this person they trust so much is deceiving them or so wrong about something important
• they sense it will damage the relationship if they refuse to believe, and they care deeply about the relationship
• they have a hard time saying ‘no’ – perhaps it makes them very anxious to do so

In other words, there are a great many dumb things that even smart people end up believing in simply because people believe people. To be clear, this is not the only mechanism by which smart people fall for dumb things. Being smart is not the same as acting rationally all the time. But this trust-based force is, I think, an important mechanism.

While a belief in others is wonderful and admirable in many instances, it can also be a chink in our skepticism and rationality. It can lead us to believe in crazy and dangerous things that we wouldn’t be likely to believe without that trust. We see this when people get scammed by their favorite influencer or when they become true believers in quack cures because they have a friend who says it changed their life.

While this effect often happens when one person we trust causes us to believe in X, the effect is magnified when more people around us believe. Being recruited into a harmful cult by a trusted friend can be difficult, but leaving a cult – at which point all of our close friends are believers in X – is far more difficult. And growing up in an authoritarian regime – where EVERYONE we’ve met seems to believe in X, makes X that much more impossible to resist.

When rationality is discussed, it’s often talked about at the level of the individual. But quite a bit of our thinking we necessarily outsource to others – we can’t make sense of everything ourselves. When we allow someone into our circle of trust who doesn’t deserve to be there, that can jeopardize our rationality. Hence, an important meta-skill of rationality is knowing who to trust – and not being suckered into trusting those who don’t deserve it.

Almost everyone is susceptible to this phenomenon of being duped because of our trust in people, but that doesn’t mean there’s nothing we can do to avoid it.

One thing that I think helps is to treat trust as being multi-factor. I can trust a person in one way or in one domain but not another. Or, put another way, earning trust is multi-dimensional. I can see someone as trustworthy because:

• I know they wouldn’t betray me and that they care a lot about me
• I know that they vet evidence carefully, come to their beliefs in a rigorous way, and approach new information skeptically
• I know that they are extremely knowledgeable about a specific topic area

Being strong in one of these domains doesn’t automatically make someone strong in another. So, viewing someone as trustworthy in one of these domains shouldn’t cause you to view them as trustworthy in the other ones. And yet, that’s what many people do.

If you track trust in a one-dimensional way, it puts you at a lot of risk because someone you trust may have a very bad idea that they really want you to believe in. It may be hard to reject that idea because you trust them so much – and that may mean joining a harmful cult, buying into the peak of the next bubble, putting stock in an ineffective treatment, or being scammed.

This piece was first written on September 13, 2024, and first appeared on my website on September 22, 2024.

How to do an Instantaneous Experiment:

Step 1: Think of anything at all about the world that’s checkable that is likely to be true if your hypothesis is true, but that is likely to be false if your hypothesis is false.

Important: this checkable thing should be something that you have never investigated before – in other words, you don’t actually know if it’s true, and the only real reason you think it’s true is just because your hypothesis implies it would be. This is critical to help prevent bias from occurring during the process (for instance, this procedure doesn’t work if the fact you are checking is one that influenced your development of the hypothesis).

Step 2: Go check whether the checkable thing is true or not by trying to look the answer up (e.g., in an article or paper)!

The amount of evidence that the answer provides in favor (or against) your hypothesis precisely depends on how many times more likely you are to see that result if your hypothesis is true compared to if it’s not true. The bigger that number is, the greater the evidence!

Instantaneous Experiments work because, to get evidence for a theory or hypothesis, it is not necessary to directly check whether that thing is true. All you have to do is check something that is implied by that theory (that would be unlikely to be true otherwise).

Here’s an example:

Suppose you believe that “greater intelligence causes people to worry a lot more”

That’s very hard to test. But you can do an Instantaneous Experiment:

Step 1: if intelligence causes worry, then you might expect higher IQ people to agree more often with a statement like “I worry too much,” whereas if the theory is not true, you wouldn’t expect a positive correlation between IQ and agreement with that statement.

Step 2: We go check this, and we find a paper that measures both IQ and the level of agreement on the statement “I worry too much.” The correlation between them is essentially 0.

Result: We haven’t completely disproven the theory, but we should now reduce our confidence in it compared to what we thought before.

How much we reduce our confidence depends on how many times less likely we’d be to find no correlation between self-reported worry and IQ if our hypothesis “greater intelligence causes people to worry a lot more” is true, compared to if it’s false.

This piece was first written on August 24, 2024, and first appeared on my website on October 11, 2024.

How can you tell who is a valid expert, and who is full of B.S.?

On almost any topic of importance you can find a mix of valid experts (who are giving you reliable information) and false but confident-seeming “experts” (who are giving you misinformation). To make matters even more confusing, sometimes the fake experts even have very impressive credentials, and every once in a while, the real, genuine experts are entirely self-taught.

Here are 12 signs we look for in an expert to help us determine whether they are trustworthy. 

1. They have deep factual knowledge

Let’s start with the obvious: for most topics, a lot of factual knowledge is required before you can have genuine expertise. This means that a genuine expert will have an impressive command of the relevant (non-debated) facts on the topic of their expertise. Thankfully, it’s a lot easier to tell if an expert has a strong command of the non-debated facts than whether they are correct about more controversial claims. 

2. They communicate their confidence levels

Not all knowledge is equally well-established. Even theories that are widely accepted enjoy different levels of support from the relevant evidence. When an expert regularly pretends that all their claims are equally well-established, they demonstrate they are willing to make you believe something is certain when it isn’t.

It’s a good sign that someone treats their subject with the nuance expected from genuine expertise, when they indicate how confident they are (e.g., “It’s been shown in many high-quality studies that…”, or “My best guess is…”), and they explain limitations in the evidence they are using (e.g., “this is unfortunately based on just one study, but that is all that currently exists”)

3. They admit not knowing

Genuine experts also sometimes say that they don’t know the answer to a question, or that the answer is generally not known by anyone. This is important because every topic will have some unknowns, and no expert can know everything about a topic. Telling you when they don’t know is a sign that, when they say they do know, they actually do know.

4. They tell you to look at sources other than themselves

This might happen when an expert doesn’t know the answer to a question, or when they want to help you go beyond the answer they can give you. Genuine experts don’t seek to be seen as a sole arbiter of knowledge or authority on a topic (which can be an indication that ego, rather than truth-seeking, is a primary motivation for them), but instead encourage you to look at resources other than the ones they have produced.

5. They use logic and evidence

Anyone can use rhetorical devices like emotional appeals, no matter how wrong they are, but a well-reasoned argument that uses valid logic and strong evidence will tend to point toward truth. Or, put another way, using strong logic and strong evidence is easier to do when you’re right, whereas emotional appeals are no easier when you’re right than when you’re wrong.  

6. They cite high-quality evidence

Some evidence is much more reliable than other evidence, and those who rely on the less reliable kinds when the more reliable kinds exist probably aren’t doing the best job they can at figuring out the truth. For this reason, genuine experts cite high-quality evidence when it exists (e.g., looking at multiple randomized controlled trials for causal claims) rather than low-quality evidence (e.g., just talking about personal anecdotes), and when high-quality evidence doesn’t exist, they cite the highest quality evidence that does exist.

7. They acknowledge the consensus

Consensus views among experts are more often correct than the idiosyncratic views of just one or two experts. The consensus will not always be right, of course, but often it will be the best understanding we have available. That’s why reliable experts are transparent about the degree to which their opinion differs from the majority of experts, provide reasoned explanations for any deviations, and they are cautious not to present fringe theories as mainstream. This shows a deep engagement with the topic of their expertise and also an adherence to ethical standards of honesty and accuracy in communication.

8. They change their mind

Genuine experts will change their minds about topics within their expertise in response to evidence and arguments. It’s hard to become an expert in something without having been wrong from time-to-time.

That means that anyone claiming to be an expert who has never changed their mind probably has not found and corrected their mistakes. Relatedly, changing one’s mind in response to evidence is also a sign of the epistemic humility associated with genuine expertise.

Of course, if someone has a long history of being wrong, that is evidence against them being a genuine expert, not in favor of it. But, since everyone makes some mistakes, if they make mistakes from time to time and then note they were wrong and improve their beliefs, that is a sign that they are following the evidence where it leads rather than continuing to believe what they do regardless of the evidence.

9. They Steelman

When you ‘straw man’ an argument, you misrepresent or oversimplify someone else’s position to make it easier to attack or refute. Instead of dealing with the actual argument, you replace it with a weaker version that distorts the original point, which you then argue against. The opposite of this is called ‘steelmanning’, and it involves presenting the strongest possible version of an argument you’re objecting to, even if it’s more robust than the one originally presented. This approach aims to strengthen the opposing case in order to facilitate a more genuine and constructive debate. 

The most reliable experts will accurately present the strongest arguments made by those that disagree with them while pointing out flaws in those arguments, rather than focusing on just weak arguments from the other side or just mocking the other side (including ad hominem attacks rather than focusing on the substance of the claims of the other side). This is important because knocking down a weak argument from the other side of a debate does little to show the other side is wrong; you have to refute the strongest claims of the other side to actually show they are wrong. Additionally, demonstrating a knowledge of the strongest arguments against your own position shows a deeper level of expertise than only understanding the opposing point of view at a superficial level.

10. They clearly explain their reasons for believing

The philosopher Daniel Dennett has said: “if I can’t explain something I’m doing to a group of bright undergraduates, I don’t really understand it myself.” This sentiment is echoed by philosopher John Searle, who said “In general, I feel if you can’t say it clearly you don’t understand it yourself.” 

When communicating with non-experts, genuine experts are often able to give clear, easy-to-follow (and, ideally, checkable) explanations for why they believe what they believe – without dumbing down the points. They avoid unnecessary jargon and technical language (which sounds smart but makes their arguments very difficult for their audience to follow). Not every genuine expert is able to do this, but the ability to do this well is a sign of genuine expertise. This is important because an expert who cannot explain their ideas clearly will end up requiring you to believe them based on their authority rather than engaging with the arguments themselves. And sometimes, people claiming to be experts will hide behind technical expertise and jargon so that you won’t notice that their arguments are actually weak.

11. They have a track record

Sometimes genuine experts will have track records of predictions or successes that you can check, and this provides direct evidence of their knowledge or skill. Unfortunately, this only applies to some fields, like chess masters, martial experts who fight in tournaments, experts who make public predictions about the economy or politics, etc.

12. They use multiple lenses

The world is complex and multi-faceted, and any one simple theory is going to fail to explain a lot of what’s really going on. For this reason, genuine experts tend to look at problems from multiple frames and perspectives; they don’t act as though one way of looking at things solves all problems, or that one solution works for all problems, or that one simple theory explains everything.

So the next time you hear claims from an alleged expert on a topic that is important to you, you may want to consider: how many of these signs of expertise do they exhibit? You can use this checklist, considering if they:

1. have deep factual knowledge
2. communicate their confidence levels
3. admit not knowing
4. tell you to look at sources other than themselves
5. use logic and evidence
6. cite high-quality evidence
7. acknowledge the consensus
8. change their mind
9. steelman
10. clearly explain their reasons for believing
11. have a track record
12. use multiple lenses

And if you’re seeking to be an expert in something yourself, you may want to ask yourself: “to what extent do I exhibit these traits?”Being able to discern genuine expertise from B.S. requires good judgment. If you’d like to improve your skills at making accurate judgments, why not try our Calibrate Your Judgment tool, created in partnership with Open Philanthropy.

This piece first appeared on Clearer Thinking.org on April 16, 2024, and first appeared on my website on April 22, 2024.

1) Belief because you think something’s true.

For instance, you may think that the evidence supports the idea that you will eventually find love, or you may feel convinced by logical arguments you’ve heard in favor of god’s existence.

2) Belief because you think it’s useful to believe. 

Regardless of whether you predict something’s true, you can predict that believing it will be more helpful than harmful to you in the long term, and so be motivated to believe for that pragmatic benefit.

For instance, you may intuit that you’ll be better off long-term believing that you will eventually find love (because that will make love more likely) or perceive that you’ll be happier believing in god (even if it turns out there is no god).

3) Belief because it feels good in the moment. 

Regardless of whether it’s true or helpful to you in the long term, you may be motivated to believe something because it feels good right now (or prevents you from feeling bad). 

For instance, you may feel comforted right now by thinking you’ll eventually find love or feel good in the moment, believing a god is watching over you.

Rationalists typically recommend striving to have your beliefs be of type 1: believing based on what’s most likely to be true.

Pragmatists often recommend aiming for type 2 beliefs: believing based on what’s ultimately most useful to you.

I favor striving to have type 1 beliefs rather than type 2 beliefs, in part because I intrinsically value truth, but also because I think that for beliefs in category 2 that are *not* actually true, there are typically some beliefs in category 1 that will help you just as much, but which have the advantage of also being true. So often (but not always), there is a low cost to replacing beliefs from 2 with beliefs from 1 that have the added benefit of being true.

I also think that if you allow yourself to indiscriminately hold type 2 beliefs, it makes it hard to suddenly switch to rigorous truth-oriented thinking when it’s important to figure out the truth (e.g., when you have to make a very important decision based on evidence).

On the other hand, many people have lots of type 3 beliefs, and all of us, myself included, have some type 3 beliefs. Whether you think that type 1 or type 2 beliefs are ultimately preferable, I think a valuable aspiration is to replace some of our type 3 beliefs with either 1s or 2s.

It’s very, very easy for us humans to delude ourselves based on what it feels good to believe at the moment because the reward cycle is so fast. Type 3 beliefs are immediately rewarding, incentivizing more such beliefs. But they are like the social media addiction version of believing, where you pursue what gives the greatest instantaneous reward rather than what’s actually good for you.

This piece was first written on April 20, 2024, and first appeared on my website on May 7, 2024.

1. Tautologies are true by definition, ’cause tautologies are true by definition.

2. Induction worked in the past, so it probably will in the future.

3. If deduction solves your problem, and you want it solved, then you’ll want to use deduction!

4. If you thought Bayesianism had 3:1 odds, and you think this sentence is 2x more likely if Bayesianism than if not Bayesianism, now you should give it 6:1 odds.

5. If frequentism wasn’t true, you’d have a low probability of reading a sentence as extreme as this one.

6. You should trust your gut because your gut is telling you to trust it.

7. The theory that Occam’s Razor is true is simpler than the theory that it is false.

8. The best explanation for why humans use abduction often is because it is useful for helping them figure out the truth.

If you liked this piece, you may also like Twelve Recursive Explanations.

This piece was first written on November 26, 2020, and first appeared on this site on August 12, 2022.

Use my lists of “Idea-Inducing Questions” to generate nearly endless ideas to write about, think about, or discuss!

Questions about learning and truth-seeking

• Recently learned: what’s a powerful idea, concept, or mental model that you’ve been learning about recently that you think is worth knowing?

• Changed opinions: what is a strongly held belief you used to have that you changed your mind about? What caused you to change your mind? Why do you think you were wrong before?

• Influential book: what is the name of one book that substantially influenced the way you think about things? What did you learn from it that you can pass on to your audience?

• Debates: what’s something you disagree with a certain group of people on? What do you think is the core of the disagreement?

• Thorny problems: Is there a complex problem, situation, idea, concept, or a set of competing ideas that you’re still trying to understand or figure out your opinion on? What are the factors that are driving your opinion in different directions or that make the issue tricky to figure out? What are the open questions or confusions you have about it still?

• Third perspectives: For any pair of opposing ideas that most people in the public sphere take either one side or the other on, can you think of a third perspective or synthesis of both ideas that could actually be better than taking either side?

• Misconceptions: What’s a commonly believed idea that you think is actually wrong or a misconception?

• Underrated or overrated ideas: What’s a powerful or useful idea that you think is significantly underrated? Or conversely, what’s an idea that is talked about a lot in a positive light that you think is overrated or that isn’t actually a good idea?

• Epistemics: how do you think about what to believe versus what ideas to reject? How do you approach understanding hotly-contested, thorny, or highly complex topics? What mental models or approaches do you use to help you think more clearly or analyze questions or evaluate evidence?

Questions about ideas that have useful applications

• Beneficial ideas: what’s an idea that, if it became widely known and adopted/used, would greatly improve the world?

• Versatile ideas: what powerful idea or concept do you think has many different useful applications across many life domains?

• Psychology tools: what’s a powerful idea, concept, mental model, or tool from psychology that you think is useful to people’s lives?

• Tools for making sense: what’s a powerful idea, concept, mental model, or tool that you think can help people better understand or make sense of the world?

• Scientific principles: what principle from a mathematical or scientific field (e.g., economics, statistics, evolutionary biology, etc.) do you think is important or valuable to know about (because it helps you understand the world or because there are applications of it to daily life)? How can you apply this idea in life?

Questions about your own ideas and experiences

• Your ideas: What’s an idea you’ve come up with that you think would be valuable for your audience to hear about?

• Ideas you’ve applied: What’s an idea that you’ve found to be very useful or powerful in your own life? How have you applied it successfully?

• Unique experiences: what’s something you have experienced that very few people have experienced (whether it’s a good thing, a bad thing, or just something strange or surprising)? What did you learn from that experience?

• On your mind: What’s an idea you’ve been thinking about a lot lately? What are your current thoughts about it?

• Area of expertise: what topic are you very knowledgeable about? What is the most valuable idea from that field that you think many people would benefit from knowing about?

Questions about unusual or neglected ideas

• Powerful obscure ideas: In your opinion, what is one of the most valuable or important ideas or concepts that most people don’t know about?

• Contrarian ideas: What’s something you disagree with most smart, educated people about (according to your own definition of smart and educated)? Or what’s your answer to the Thiel question: “What important truth do very few people agree with you on?”

• Overlooked areas: What’s a topic area that very few people have an opinion on or knowledge of, that you think it’s important to have an opinion on, or that is well worth learning about?

This piece was first written on August 2, 2020, and first appeared on this site on May 27, 2022.