How to do an Instantaneous Experiment:

Step 1: Think of anything at all about the world that’s checkable that is likely to be true if your hypothesis is true, but that is likely to be false if your hypothesis is false.

Important: this checkable thing should be something that you have never investigated before – in other words, you don’t actually know if it’s true, and the only real reason you think it’s true is just because your hypothesis implies it would be. This is critical to help prevent bias from occurring during the process (for instance, this procedure doesn’t work if the fact you are checking is one that influenced your development of the hypothesis).

Step 2: Go check whether the checkable thing is true or not by trying to look the answer up (e.g., in an article or paper)!

The amount of evidence that the answer provides in favor (or against) your hypothesis precisely depends on how many times more likely you are to see that result if your hypothesis is true compared to if it’s not true. The bigger that number is, the greater the evidence!

Instantaneous Experiments work because, to get evidence for a theory or hypothesis, it is not necessary to directly check whether that thing is true. All you have to do is check something that is implied by that theory (that would be unlikely to be true otherwise).

Here’s an example:

Suppose you believe that “greater intelligence causes people to worry a lot more”

That’s very hard to test. But you can do an Instantaneous Experiment:

Step 1: if intelligence causes worry, then you might expect higher IQ people to agree more often with a statement like “I worry too much,” whereas if the theory is not true, you wouldn’t expect a positive correlation between IQ and agreement with that statement.

Step 2: We go check this, and we find a paper that measures both IQ and the level of agreement on the statement “I worry too much.” The correlation between them is essentially 0.

Result: We haven’t completely disproven the theory, but we should now reduce our confidence in it compared to what we thought before.

How much we reduce our confidence depends on how many times less likely we’d be to find no correlation between self-reported worry and IQ if our hypothesis “greater intelligence causes people to worry a lot more” is true, compared to if it’s false.

This piece was first written on August 24, 2024, and first appeared on my website on October 11, 2024.

I think that almost everyone has beliefs that are essentially unchangeable. These don’t feel to us like beliefs but like incontrovertible truths. Counter-evidence can’t touch them. They are beliefs we can’t change our mind about. I call these “Anchor Beliefs.”

When Anchor Beliefs are false, we distort reality to fit them. So, what distortions do some reasonably common Anchor Beliefs cause?

Anchor Belief 1: “I’m entirely good” or “I don’t do unethical things”

What happens when someone with these Anchor Beliefs acts highly unethically? Well, since the Anchor Belief can’t change, that means the action must have been ethically okay to do, or else it was someone else’s fault or impossible to avoid. Victim blaming, denial, or shirking of responsibility ensues.

“My whole foundation, life, what I believed in, devotion to the company, was based on believing [Ramesh Balwani] was this person…He told me he didn’t know what I was doing in business, that my convictions were wrong…There was no way I could save our company if he was there…We were trying to do the right thing. We were trying to report results that we believed in and not report results if we thought there was any issue” -Elizabeth Holmes, who was found guilty on four counts of defrauding the investors in her company, Theranos

“All I ever wanted was to love women and, in turn, to be loved by them back. Their behavior towards me has only earned my hatred, and rightfully so! I am the true victim in all of this. I am the good guy.” -Elliot Rodger, in his manifesto about why he planned to commit murder before murdering six people.

Anchor Belief 2: “I’m not good enough”

What happens when someone with this anchor belief gets a great job, performs really well, or achieves success? Well, it must have been a fluke or mistake; eventually, others will figure it out. Imposter syndrome ensues.

“No matter what we’ve done, there comes a point where you think, ‘How did I get here? When are they going to discover that I am, in fact, a fraud and take everything away from me?” – Tom Hanks, winner of two consecutive Academy Awards for Best Actor

“I have written 11 books, but each time I think, ‘Uh oh, they’re going to find out now. I’ve run a game on everybody, and they’re going to find me out.” – Maya Angelou, legendary poet and winner of the Presidential Medal of Freedom.

Anchor Belief 3: “This thing I’ve devoted a great deal of time/energy/identity into works and is good” [that doesn’t work or is harmful]

What happens when it’s criticized? The criticism must be bad faith. Any imperfection in counter-evidence fully invalidates that evidence. Confirmation bias, cherry-picking, and motivated reasoning ensues.

“Those who have attacked my work on Vitamin C are scoundrels.” – Linus Pauling, two-time Nobel prize winner, defending his theory that vitamin C cures cancer and heart disease.

“We do not find critics of Scientology who do not have criminal pasts…Politician A stands up on his hind legs in a Parliament and brays for a condemnation of Scientology. When we look him over we find crimes – embezzled funds, moral lapses, a thirst for young boys – sordid stuff. Wife B howls at her husband for attending a Scientology group. We look her up and find she had a baby he didn’t know about.” – L. Ron Hubbard, founder of Scientology

Anchor Belief 4: “My group is good.”

What happens when their group does something really bad? The victims must be lying or have deserved it. Or acting badly must be justified in this case because it’s done for some more important greater good. Denial of and justification of immoral actions ensues.

“When we show a statement by Donald Trump that’s not truthful, Republicans will say it’s okay if it’s not true because it sends the right message, whereas Democrats will say that a statement needs to be factual…With a statement from Joe Biden, Democrats will say it’s okay if it’s not based on evidence, that it supports a generally true message, while Republicans will then have a higher bar and say every statement needs to be based on facts.” – Ethan Poskanzer, based on his studies on moral flexibility

So, what are the takeaways here? I think that the following three things are important and true:

(1) Almost everyone has at least one Anchor Belief – a belief that is so sticky that it’s nearly impossible for it to change in the face of even extremely strong counter-evidence. Some people have more of these, and perhaps a small number of people have none, but I think Anchor beliefs are a near-universal among us humans.

(2) When our Anchor Beliefs are false (or partially false), because the beliefs won’t change, we distort reality when we get evidence against them in order to keep them intact while also somehow “making sense” of that counter-evidence.

(3) By looking at fairly common Anchor Beliefs people have, we can start to understand some recurring distortions in people’s thinking. Since people’s Anchor Beliefs are fixed but reality sometimes provides strong counter-evidence against these beliefs, that leads to predictable patterns of distortions that people’s minds deploy to keep the beliefs intact around those Anchor Beliefs.

In particular, I think that we find:

• Anchor Beliefs related to being good may lead to victim blaming and denial of responsibility.

• Anchor Beliefs about not being good enough may lead to imposter syndrome.

• Anchor Beliefs about something we’ve invested a lot of time/energy/identity into working on and being good may lead to confirmation bias, cherry-picking, and motivated reasoning.

• Anchor Beliefs about our group being good may lead us to deny or justify immoral actions by our group.

There are no strong studies that I’m aware of that identify or map out anchor beliefs and their frequency in the population – I believe the points above are true based on my experiences and observations.

This piece was first written on August 13, 2024, and first appeared on my website on September 2, 2024.

How can you tell who is a valid expert, and who is full of B.S.?

On almost any topic of importance you can find a mix of valid experts (who are giving you reliable information) and false but confident-seeming “experts” (who are giving you misinformation). To make matters even more confusing, sometimes the fake experts even have very impressive credentials, and every once in a while, the real, genuine experts are entirely self-taught.

Here are 12 signs we look for in an expert to help us determine whether they are trustworthy. 

1. They have deep factual knowledge

Let’s start with the obvious: for most topics, a lot of factual knowledge is required before you can have genuine expertise. This means that a genuine expert will have an impressive command of the relevant (non-debated) facts on the topic of their expertise. Thankfully, it’s a lot easier to tell if an expert has a strong command of the non-debated facts than whether they are correct about more controversial claims. 

2. They communicate their confidence levels

Not all knowledge is equally well-established. Even theories that are widely accepted enjoy different levels of support from the relevant evidence. When an expert regularly pretends that all their claims are equally well-established, they demonstrate they are willing to make you believe something is certain when it isn’t.

It’s a good sign that someone treats their subject with the nuance expected from genuine expertise, when they indicate how confident they are (e.g., “It’s been shown in many high-quality studies that…”, or “My best guess is…”), and they explain limitations in the evidence they are using (e.g., “this is unfortunately based on just one study, but that is all that currently exists”)

3. They admit not knowing

Genuine experts also sometimes say that they don’t know the answer to a question, or that the answer is generally not known by anyone. This is important because every topic will have some unknowns, and no expert can know everything about a topic. Telling you when they don’t know is a sign that, when they say they do know, they actually do know.

4. They tell you to look at sources other than themselves

This might happen when an expert doesn’t know the answer to a question, or when they want to help you go beyond the answer they can give you. Genuine experts don’t seek to be seen as a sole arbiter of knowledge or authority on a topic (which can be an indication that ego, rather than truth-seeking, is a primary motivation for them), but instead encourage you to look at resources other than the ones they have produced.

5. They use logic and evidence

Anyone can use rhetorical devices like emotional appeals, no matter how wrong they are, but a well-reasoned argument that uses valid logic and strong evidence will tend to point toward truth. Or, put another way, using strong logic and strong evidence is easier to do when you’re right, whereas emotional appeals are no easier when you’re right than when you’re wrong.  

6. They cite high-quality evidence

Some evidence is much more reliable than other evidence, and those who rely on the less reliable kinds when the more reliable kinds exist probably aren’t doing the best job they can at figuring out the truth. For this reason, genuine experts cite high-quality evidence when it exists (e.g., looking at multiple randomized controlled trials for causal claims) rather than low-quality evidence (e.g., just talking about personal anecdotes), and when high-quality evidence doesn’t exist, they cite the highest quality evidence that does exist.

7. They acknowledge the consensus

Consensus views among experts are more often correct than the idiosyncratic views of just one or two experts. The consensus will not always be right, of course, but often it will be the best understanding we have available. That’s why reliable experts are transparent about the degree to which their opinion differs from the majority of experts, provide reasoned explanations for any deviations, and they are cautious not to present fringe theories as mainstream. This shows a deep engagement with the topic of their expertise and also an adherence to ethical standards of honesty and accuracy in communication.

8. They change their mind

Genuine experts will change their minds about topics within their expertise in response to evidence and arguments. It’s hard to become an expert in something without having been wrong from time-to-time.

That means that anyone claiming to be an expert who has never changed their mind probably has not found and corrected their mistakes. Relatedly, changing one’s mind in response to evidence is also a sign of the epistemic humility associated with genuine expertise.

Of course, if someone has a long history of being wrong, that is evidence against them being a genuine expert, not in favor of it. But, since everyone makes some mistakes, if they make mistakes from time to time and then note they were wrong and improve their beliefs, that is a sign that they are following the evidence where it leads rather than continuing to believe what they do regardless of the evidence.

9. They Steelman

When you ‘straw man’ an argument, you misrepresent or oversimplify someone else’s position to make it easier to attack or refute. Instead of dealing with the actual argument, you replace it with a weaker version that distorts the original point, which you then argue against. The opposite of this is called ‘steelmanning’, and it involves presenting the strongest possible version of an argument you’re objecting to, even if it’s more robust than the one originally presented. This approach aims to strengthen the opposing case in order to facilitate a more genuine and constructive debate. 

The most reliable experts will accurately present the strongest arguments made by those that disagree with them while pointing out flaws in those arguments, rather than focusing on just weak arguments from the other side or just mocking the other side (including ad hominem attacks rather than focusing on the substance of the claims of the other side). This is important because knocking down a weak argument from the other side of a debate does little to show the other side is wrong; you have to refute the strongest claims of the other side to actually show they are wrong. Additionally, demonstrating a knowledge of the strongest arguments against your own position shows a deeper level of expertise than only understanding the opposing point of view at a superficial level.

10. They clearly explain their reasons for believing

The philosopher Daniel Dennett has said: “if I can’t explain something I’m doing to a group of bright undergraduates, I don’t really understand it myself.” This sentiment is echoed by philosopher John Searle, who said “In general, I feel if you can’t say it clearly you don’t understand it yourself.” 

When communicating with non-experts, genuine experts are often able to give clear, easy-to-follow (and, ideally, checkable) explanations for why they believe what they believe – without dumbing down the points. They avoid unnecessary jargon and technical language (which sounds smart but makes their arguments very difficult for their audience to follow). Not every genuine expert is able to do this, but the ability to do this well is a sign of genuine expertise. This is important because an expert who cannot explain their ideas clearly will end up requiring you to believe them based on their authority rather than engaging with the arguments themselves. And sometimes, people claiming to be experts will hide behind technical expertise and jargon so that you won’t notice that their arguments are actually weak.

11. They have a track record

Sometimes genuine experts will have track records of predictions or successes that you can check, and this provides direct evidence of their knowledge or skill. Unfortunately, this only applies to some fields, like chess masters, martial experts who fight in tournaments, experts who make public predictions about the economy or politics, etc.

12. They use multiple lenses

The world is complex and multi-faceted, and any one simple theory is going to fail to explain a lot of what’s really going on. For this reason, genuine experts tend to look at problems from multiple frames and perspectives; they don’t act as though one way of looking at things solves all problems, or that one solution works for all problems, or that one simple theory explains everything.

So the next time you hear claims from an alleged expert on a topic that is important to you, you may want to consider: how many of these signs of expertise do they exhibit? You can use this checklist, considering if they:

1. have deep factual knowledge
2. communicate their confidence levels
3. admit not knowing
4. tell you to look at sources other than themselves
5. use logic and evidence
6. cite high-quality evidence
7. acknowledge the consensus
8. change their mind
9. steelman
10. clearly explain their reasons for believing
11. have a track record
12. use multiple lenses

And if you’re seeking to be an expert in something yourself, you may want to ask yourself: “to what extent do I exhibit these traits?”Being able to discern genuine expertise from B.S. requires good judgment. If you’d like to improve your skills at making accurate judgments, why not try our Calibrate Your Judgment tool, created in partnership with Open Philanthropy.

This piece first appeared on Clearer Thinking.org on April 16, 2024, and first appeared on my website on April 22, 2024.

Some remarkable things I suspect are true of nearly all adults: 

1) We each hold some beliefs that are almost totally non-responsive to evidence involving some combination of our identity (who we are), our group, the nature of reality (e.g., God), or the nature of what’s good.

Examples:

• Many have an unshakable belief that they are good even as they harm the world (or believe they’re insufficient even though they’re altruistic and productive)

• Most have an unshakable belief that their in-group is good and any group opposing their group is bad

2) We assume that other people’s internal experiences are more similar to our internal experiences than they really are. Consequently, we tend to predict they’ll behave more like us than they really will.

Example:

• You’re an anxious person who avoids situations you’re afraid of, so you predict other people will be more afraid of similar situations than they really will be and that they’ll be more avoidant than they really will be

3) Emotions alter our behaviors and thoughts (increasing the likelihood of some behaviors and thoughts, decreasing the likelihood of others) in emotion-dependent ways.

Examples: 

• Physical disgust increases the chance of backing away and reduces the chance of eating soon

• Feelings of depression increase the chance of thinking thoughts about situations being hopeless or actions being pointless

4) How good or bad we feel about something happening depends on the difference between our expectations about what will happen and the reality of what actually happened.

Examples: 

• If you think someone with a gun is about to shoot, but instead, they take your wallet and run, you might feel relief (whereas normally wallet theft would be highly distressing) 

• If you expect to make $300k on a deal, you might feel bad if you “only” get $200k

5) We have multiple “drives” encoded in our brains that want different things (e.g., they have different values or goals), and these often come into conflict. Our behavior is influenced by which drives are activated and how strongly each is activated.

Examples: 

• If we smell delicious popcorn right in front of us, most of us will eat it, whereas if it’s a few feet away and we can’t smell it, we’re less likely to 

• If we’re exhausted but also slightly hungry, we may delay making food until we are more hungry or less tired

6) We are influenced by the behavioral norms and patterns demonstrated by the people around us, especially when they are people who we identify as being part of our group or similar to us.

Example: 

• If it’s common to dress or talk a certain way in a place we move to, it will increase the chance we start to dress and talk similarly

This piece was first written on October 20, 2023, and first appeared on my website on February 7, 2024.

(1) Linus Pauling won TWO Nobel prizes – one in peace and one in chemistry. Unfortunately, he eventually became obsessed with and widely promoted the false (and sometimes still repeated) idea that high-dose vitamin C cures many diseases, including HIV and snakebites.

(2) Isaac Newton, who co-invented calculus and discovered the laws of gravity, also was convinced the Bible had hidden messages he could decode for prophetic purposes, and spent a lot of time trying to create the mythical philosopher’s stone, so he could turn metal into gold.

(3) Alan Turing, often considered to be the father of theoretical computer science and artificial intelligence, seemingly was convinced by the existence of extrasensory perception. He wrote: “the statistical evidence, at least for telepathy, is overwhelming.”

(4) C. Langan, who appears to have one of the highest IQs ever recorded, believes “you can prove the existence of God, the soul, and an afterlife, using mathematics.” and has claimed that 9/11 was an inside job staged by the Bush administration.

My point is not that these people were stupid – they are the opposite of stupid – they are far smarter than 99.9% of the people that have ever lived (by at least some reasonably common ways of thinking about intelligence). My point is that even the smartest among us hold some silly, false beliefs – intelligence is not enough to avoid error.

Rationality (in the sense of evaluating evidence in such a way as to effectively arrive at the truth on important topics) and intelligence, while related, are also not the same things. Rationality involves actively working to disprove your own beliefs – which intelligent people may or may not do. For instance, intelligence is often used to come up with clever and convincing arguments for why what you already think must indeed be correct. In other words, intelligence can be deployed for rationality but also for rationalization.

Of course, it may also be me that’s wrong. Perhaps there’s a philosopher’s stone, vitamin C cures a ton of diseases, 9/11 was an inside job. But more likely, I’m wrong about other things (that I have no clue I’m wrong about). It’s useful to remember: we all believe false things.

This piece was first written on July 16, 2023, and first appeared on this site on August 16, 2023.

What matters (evidence-wise) is how likely that level of improvement would have been in that time period if the treatment works relative to how likely that improvement would have been if the treatment is useless.

For something like tiredness, which tends to fluctuate a lot, feeling somewhat less tired than normal after two weeks may provide almost no evidence a treatment worked. But if you feel less tired than you have in 10 years, that could be strong evidence!

To give another example, if you’ve had a rash without a break for years, and the rash goes away in one day with a new cream, that is very strong evidence the cream worked. But if the rash very often comes and goes on its own, or it took six months of using the cream before it disappeared, its disappearance provides little evidence of effectiveness.

More formally, the amount of evidence an improvement gives you (in favor of the treatment working) is:

Bayes Factor = the probability that you’d see this level of improvement given that the treatment works / the probability that you’d see this level of improvement given that the treatment doesn’t work

In words, this is just “how many times more likely is it that you’d see this level of improvement during this period of time if the treatment works compared to if it doesn’t work.”

This Bayes Factor is what you multiply your prior odds by. So if, before trying the treatment, you thought there were 1 to 3 odds of it working (i.e., a 25% chance), and if you now you get a Bayes factor of 6, you should now believe there are 6*(1/3) = 2 to 1 odds that it works (i.e., a 66% chance).

While it’s rare to be able to do this calculation precisely, it’s this general way of thinking (in terms of relative likelihoods, comparing a world where the treatment works to one where it doesn’t) that’s important. I find this to be an especially helpful application of Bayes’ rule which can guide practical decision-making (e.g., whether to stick with a new treatment).

This piece was first written on April 15, 2023, and first appeared on this site on August 2, 2023.

1. Soldier Altruists think it’s obvious how to improve the world and that we just need to execute those obvious steps. They see the barriers to a better world as:

(i) not enough people taking action (e.g., due to ignorance, selfishness, or propaganda), and

(ii) bad groups blocking things (e.g., corrupt politicians or greedy corporations).

2. Scout Altruists think it’s hard to figure out how to improve the world – and most attempts either don’t work, only slightly help, or make things worse. They see the barriers to a better world as:

(i) not enough understanding of causal mechanisms (e.g., due to a lack of high-quality evidence, not enough attention to the evidence we do have, not enough careful reasoning, etc.), and

(ii) too much investment in bad solutions (e.g., due to people jumping to conclusions, doing what feels good emotionally rather than what is effective, ideological blindspots, inertia, etc.)

Soldier Altruists say we need to DO and FIGHT more. Scout Altruists say we need to THINK and TEST more. Soldier Altruists are more likely to think that if we could just get people to be less selfish and more motivated to act, we would make a lot of progress towards a better world. Scout Altruists are more likely to think that if we could just get people to pay more attention to evidence and to have more good-faith debates with strong norms around the quality of argumentation, we would make a lot more progress.

Soldier Altruists may think Scout Altruists are far too reluctant to act and are wasting their time on research and debate. Scout Altruists may think Soldier Altruists are far too confident in their conclusions and are wasting their effort pushing for changes that aren’t going to help much (and which, in some cases, might even make things worse). Of course, in reality, there is a continuum between these two positions. So, on a scale from 0 (Soldier Altruist) to 10 (Scout Altruist) where do you fall? I’m probably a 7 or 8.

As some commenters have pointed out, there is a relationship between this distinction and “Conflict Theory” vs. “Mistake Theory.” I think it is related – but also distinct in important ways. Conflict theory says that there is a giant zero-sum struggle (groups fighting over fixed resources). Whereas in this case, we’re operating from a framework of altruism: “the world can be made a lot better – what’s the big barrier to that happening? Is it that we know what to do and we’re not doing it enough/with enough energy, or is it that we don’t really know what to do?”

Also, to clarify another important point brought up in the comments: I’m not asking, “do you think it’s obvious how we should improve the world if we had a magic wand that could change whatever we wanted?” – instead, the question is: “is it obvious what to do to improve the real world, given that we don’t have a magic wand?” Do we just need to put more money/time/effort/people into executing the current “obvious” strategies because they will work well if we just scale them up? Or is it pretty unclear what strategies we should even be putting more resources into (meaning that a lot of thinking, research, debate and/or evidence evaluation will typically be necessary to even figure out what is worth scaling up)?

Julia’s book (which I highly recommend): https://www.amazon.com/Scout-Mindset-Perils-Defensive-Thinking/dp/0735217556/ref=nodl_

This piece was first written on April 23rd, 2021, and first appeared on this site on January 7th, 2022.