The reason stems from my thinking that all of the following are important and true (and too often overlooked) regarding science:

1) A lot of science is real AND valuable to society.

2) A lot of “science” is actually fake – see, for instance, a decent percentage of papers in psychology 15 years ago.

3) “Science” (as an approach to knowledge discovery) is one of humanity’s greatest inventions – but in practice, it is reasonably often misapplied, or the process is distorted due to bad incentives or poor training. Unfortunately, not all fields of science have done a good job of being self-correcting either, so sometimes, fields go in bad directions for quite a while and need reform. There are different kinds of bad science:

(i) Sometimes, science is “bad” because it uses unsound methods for figuring out the truth (such as when p-hacking is rampant).

(ii) Sometimes it is “bad” because it overclaims (e.g., “Importance Hacking” where scientists claim they found something important/valuable when they didn’t actually demonstrate what they claim in their study. Or cases where science is used to “prove” questions that can’t be proven by science – such as which policy is better in a particular context when it’s actually a tradeoff between different values).

(iii) Other times science is bad because it is biased (e.g., when people are only willing to run or publish studies that show X but not that show the opposite of X).

(iv) And sometimes science is bad because it’s simply fraudulent.

4) Promoting broad “trust the science” is misguided (and actually harmful) because a bunch of science is fake. If you tell people to always just “trust the science,” then you are going to cause them to be tricked by a bunch of bad science, or you are going to contribute to their disillusionment and loss of trust when they discover (correctly) that some of the science you’re saying is good is actually garbage.

5) The “distrust all science” view is probably an even worse take than “trust the science.” If you distrust all science, you are likely to miss out on incredible things (such as highly effective treatments), and you set yourself up to fall for tons of things that don’t work (e.g., widely used unscientific treatments). Those who tell people to always just “trust the science” sometimes accidentally push people into the “distrust all science” view when those people realize that some of what they are being told to trust is crap.

6) So, hard as it is, rather than promoting either “trust all science” or “distrust all science,” the course of action I believe in with regard to science education is to teach people that “Science” (as a method) is an incredibly powerful and useful invention, but that “science” (as actually practiced) is much like every other field: some of it is good, some of it is crap. There are good hairdressers and bad hairdressers, and there is good science and bad science (and unfortunately, some bad science ends up in the very top journals – while journals and peer review absolutely do block some bad science, they unfortunately still let through quite a lot of it).

Since some science is well done, and some of it is poorly done, it’s very valuable to learn to tell the difference to make the best use of scientific results – both with regard to applying it in your own life and using it to form your beliefs about the world.

If we pretend science is all good or all bad, we do a lot of harm. We need nuance to see through the bad stuff while maintaining the tremendous benefits.

This piece was first written on November 20, 2024, and first appeared on my website on January 14, 2025.

1) Belief because you think something’s true.

For instance, you may think that the evidence supports the idea that you will eventually find love, or you may feel convinced by logical arguments you’ve heard in favor of god’s existence.

2) Belief because you think it’s useful to believe. 

Regardless of whether you predict something’s true, you can predict that believing it will be more helpful than harmful to you in the long term, and so be motivated to believe for that pragmatic benefit.

For instance, you may intuit that you’ll be better off long-term believing that you will eventually find love (because that will make love more likely) or perceive that you’ll be happier believing in god (even if it turns out there is no god).

3) Belief because it feels good in the moment. 

Regardless of whether it’s true or helpful to you in the long term, you may be motivated to believe something because it feels good right now (or prevents you from feeling bad). 

For instance, you may feel comforted right now by thinking you’ll eventually find love or feel good in the moment, believing a god is watching over you.

Rationalists typically recommend striving to have your beliefs be of type 1: believing based on what’s most likely to be true.

Pragmatists often recommend aiming for type 2 beliefs: believing based on what’s ultimately most useful to you.

I favor striving to have type 1 beliefs rather than type 2 beliefs, in part because I intrinsically value truth, but also because I think that for beliefs in category 2 that are *not* actually true, there are typically some beliefs in category 1 that will help you just as much, but which have the advantage of also being true. So often (but not always), there is a low cost to replacing beliefs from 2 with beliefs from 1 that have the added benefit of being true.

I also think that if you allow yourself to indiscriminately hold type 2 beliefs, it makes it hard to suddenly switch to rigorous truth-oriented thinking when it’s important to figure out the truth (e.g., when you have to make a very important decision based on evidence).

On the other hand, many people have lots of type 3 beliefs, and all of us, myself included, have some type 3 beliefs. Whether you think that type 1 or type 2 beliefs are ultimately preferable, I think a valuable aspiration is to replace some of our type 3 beliefs with either 1s or 2s.

It’s very, very easy for us humans to delude ourselves based on what it feels good to believe at the moment because the reward cycle is so fast. Type 3 beliefs are immediately rewarding, incentivizing more such beliefs. But they are like the social media addiction version of believing, where you pursue what gives the greatest instantaneous reward rather than what’s actually good for you.

This piece was first written on April 20, 2024, and first appeared on my website on May 7, 2024.

People often talk about what a problem “confirmation bias” is. But we rarely discuss what causes so many of us to search for information in a biased way.

Let’s explore some of the forces:

1. Echo chambers: our routine sources of information tend to support our worldview. Much of this is due to social ties (we tend to talk to people who are similar to us in age, geography, religion, etc.) We also trust news sources more if they share our basic ideology/assumptions. The authorities we look to will be the ones that agree with us on most of our fundamental assumptions (even if some of these assumptions could turn out to be wrong).

2. Soldier Mindset: as Julia Galef explains in her wonderful new book (the Scout Mindset), we are often not even TRYING to figure out the truth. We’re just trying to beat the other side or prove a point. In these cases, of course we have a biased search process.

3. Lack of doubt: when we’re really confident our basic premises are correct, we don’t see the need for a nuanced information search process. We’re going to go to whatever sources are convenient for filling in minor details, even if our beliefs have major unquestioned assumptions.

4. Advocacy: others are actively trying to get us to believe certain falsehoods. They do so through ads, websites, news, and other channels. For the most part, they themselves believe these falsehoods (promoting “the truth” as they see it), but occasionally it’s pure manipulation.

5. Fear of being wrong: it hurts to be wrong, especially if we’ve made the error publicly, have our identity tied up in the belief, or it challenges our understanding of the world or who to trust. We sometimes avoid finding out we are wrong the way we avoid touching a hot stove.

We don’t always seek out information in a biased way. For instance, when looking up driving directions or trying to figure out what paint to use to prevent water damage, we want the right answer, don’t have political biases, and usually have appropriate self-doubt and skepticism.

But we are liable to have a biased search for the truth when we are incentivized to have particular beliefs, such as when our social world supports just one perspective, when we’re trying to prove the other side wrong, when we have no doubt that we are right, when powerful others are devoting a lot of effort to persuade us, or when we’re too afraid of being wrong.

For more about this topic, you may want to check out my recent podcast episode with Julia Galef.