How can you tell who is a valid expert, and who is full of B.S.?

On almost any topic of importance you can find a mix of valid experts (who are giving you reliable information) and false but confident-seeming “experts” (who are giving you misinformation). To make matters even more confusing, sometimes the fake experts even have very impressive credentials, and every once in a while, the real, genuine experts are entirely self-taught.

Here are 12 signs we look for in an expert to help us determine whether they are trustworthy. 

1. They have deep factual knowledge

Let’s start with the obvious: for most topics, a lot of factual knowledge is required before you can have genuine expertise. This means that a genuine expert will have an impressive command of the relevant (non-debated) facts on the topic of their expertise. Thankfully, it’s a lot easier to tell if an expert has a strong command of the non-debated facts than whether they are correct about more controversial claims. 

2. They communicate their confidence levels

Not all knowledge is equally well-established. Even theories that are widely accepted enjoy different levels of support from the relevant evidence. When an expert regularly pretends that all their claims are equally well-established, they demonstrate they are willing to make you believe something is certain when it isn’t.

It’s a good sign that someone treats their subject with the nuance expected from genuine expertise, when they indicate how confident they are (e.g., “It’s been shown in many high-quality studies that…”, or “My best guess is…”), and they explain limitations in the evidence they are using (e.g., “this is unfortunately based on just one study, but that is all that currently exists”)

3. They admit not knowing

Genuine experts also sometimes say that they don’t know the answer to a question, or that the answer is generally not known by anyone. This is important because every topic will have some unknowns, and no expert can know everything about a topic. Telling you when they don’t know is a sign that, when they say they do know, they actually do know.

4. They tell you to look at sources other than themselves

This might happen when an expert doesn’t know the answer to a question, or when they want to help you go beyond the answer they can give you. Genuine experts don’t seek to be seen as a sole arbiter of knowledge or authority on a topic (which can be an indication that ego, rather than truth-seeking, is a primary motivation for them), but instead encourage you to look at resources other than the ones they have produced.

5. They use logic and evidence

Anyone can use rhetorical devices like emotional appeals, no matter how wrong they are, but a well-reasoned argument that uses valid logic and strong evidence will tend to point toward truth. Or, put another way, using strong logic and strong evidence is easier to do when you’re right, whereas emotional appeals are no easier when you’re right than when you’re wrong.  

6. They cite high-quality evidence

Some evidence is much more reliable than other evidence, and those who rely on the less reliable kinds when the more reliable kinds exist probably aren’t doing the best job they can at figuring out the truth. For this reason, genuine experts cite high-quality evidence when it exists (e.g., looking at multiple randomized controlled trials for causal claims) rather than low-quality evidence (e.g., just talking about personal anecdotes), and when high-quality evidence doesn’t exist, they cite the highest quality evidence that does exist.

7. They acknowledge the consensus

Consensus views among experts are more often correct than the idiosyncratic views of just one or two experts. The consensus will not always be right, of course, but often it will be the best understanding we have available. That’s why reliable experts are transparent about the degree to which their opinion differs from the majority of experts, provide reasoned explanations for any deviations, and they are cautious not to present fringe theories as mainstream. This shows a deep engagement with the topic of their expertise and also an adherence to ethical standards of honesty and accuracy in communication.

8. They change their mind

Genuine experts will change their minds about topics within their expertise in response to evidence and arguments. It’s hard to become an expert in something without having been wrong from time-to-time.

That means that anyone claiming to be an expert who has never changed their mind probably has not found and corrected their mistakes. Relatedly, changing one’s mind in response to evidence is also a sign of the epistemic humility associated with genuine expertise.

Of course, if someone has a long history of being wrong, that is evidence against them being a genuine expert, not in favor of it. But, since everyone makes some mistakes, if they make mistakes from time to time and then note they were wrong and improve their beliefs, that is a sign that they are following the evidence where it leads rather than continuing to believe what they do regardless of the evidence.

9. They Steelman

When you ‘straw man’ an argument, you misrepresent or oversimplify someone else’s position to make it easier to attack or refute. Instead of dealing with the actual argument, you replace it with a weaker version that distorts the original point, which you then argue against. The opposite of this is called ‘steelmanning’, and it involves presenting the strongest possible version of an argument you’re objecting to, even if it’s more robust than the one originally presented. This approach aims to strengthen the opposing case in order to facilitate a more genuine and constructive debate. 

The most reliable experts will accurately present the strongest arguments made by those that disagree with them while pointing out flaws in those arguments, rather than focusing on just weak arguments from the other side or just mocking the other side (including ad hominem attacks rather than focusing on the substance of the claims of the other side). This is important because knocking down a weak argument from the other side of a debate does little to show the other side is wrong; you have to refute the strongest claims of the other side to actually show they are wrong. Additionally, demonstrating a knowledge of the strongest arguments against your own position shows a deeper level of expertise than only understanding the opposing point of view at a superficial level.

10. They clearly explain their reasons for believing

The philosopher Daniel Dennett has said: “if I can’t explain something I’m doing to a group of bright undergraduates, I don’t really understand it myself.” This sentiment is echoed by philosopher John Searle, who said “In general, I feel if you can’t say it clearly you don’t understand it yourself.” 

When communicating with non-experts, genuine experts are often able to give clear, easy-to-follow (and, ideally, checkable) explanations for why they believe what they believe – without dumbing down the points. They avoid unnecessary jargon and technical language (which sounds smart but makes their arguments very difficult for their audience to follow). Not every genuine expert is able to do this, but the ability to do this well is a sign of genuine expertise. This is important because an expert who cannot explain their ideas clearly will end up requiring you to believe them based on their authority rather than engaging with the arguments themselves. And sometimes, people claiming to be experts will hide behind technical expertise and jargon so that you won’t notice that their arguments are actually weak.

11. They have a track record

Sometimes genuine experts will have track records of predictions or successes that you can check, and this provides direct evidence of their knowledge or skill. Unfortunately, this only applies to some fields, like chess masters, martial experts who fight in tournaments, experts who make public predictions about the economy or politics, etc.

12. They use multiple lenses

The world is complex and multi-faceted, and any one simple theory is going to fail to explain a lot of what’s really going on. For this reason, genuine experts tend to look at problems from multiple frames and perspectives; they don’t act as though one way of looking at things solves all problems, or that one solution works for all problems, or that one simple theory explains everything.

So the next time you hear claims from an alleged expert on a topic that is important to you, you may want to consider: how many of these signs of expertise do they exhibit? You can use this checklist, considering if they:

1. have deep factual knowledge
2. communicate their confidence levels
3. admit not knowing
4. tell you to look at sources other than themselves
5. use logic and evidence
6. cite high-quality evidence
7. acknowledge the consensus
8. change their mind
9. steelman
10. clearly explain their reasons for believing
11. have a track record
12. use multiple lenses

And if you’re seeking to be an expert in something yourself, you may want to ask yourself: “to what extent do I exhibit these traits?”Being able to discern genuine expertise from B.S. requires good judgment. If you’d like to improve your skills at making accurate judgments, why not try our Calibrate Your Judgment tool, created in partnership with Open Philanthropy.

This piece first appeared on Clearer Thinking.org on April 16, 2024, and first appeared on my website on April 22, 2024.

But part of what makes me an extreme non-credentialist is that if I spend an hour watching someone with a nutrition Ph.D. debate a completely self-taught person, and the Ph.D. is making bad arguments and pointing to weak evidence, and the self-taught person is making very solid arguments and pointing to strong evidence and has a very solid command of the relevant facts, the fact that the first person has a Ph.D. will be nearly completely washed out for me at that point, and I will trust the second person’s view of nutrition far more based on the quality of their thinking and the reasons underlying why they believe what they do.

So, being a non-credentialist to me isn’t about thinking that credentials are meaningless, but rather, it involves being willing to quickly update away from the evidence of a credential once you have more direct evidence about the way a person comes to conclusions and what they know.

Most Ph.D.s in a subject are vastly more reliable sources of information on that subject than most non-Ph.D.s on that same subject, but there are lots of exceptions, and sometimes self-taught people are absolutely world-class (and, in any human endeavor, plenty of people with fancy credentials are actually full of B.S.)

Another thing that makes me a non-credentialist is that I love to see highly credible, highly knowledgeable, self-taught people discussing topics and spreading their ideas (whereas some people are very much rubbed the wrong way when someone is talking publicly about a topic they lack a credential in).

An important note: when there is a strong scientific consensus, that is usually a strong starting point for beliefs on topics you know little about (e.g., in physics or biology), even though the consensus is not always right. But trusting the scientific consensus is not the same as trusting one person due to their credentials – a strong scientific consensus is typically more reliable than individual experts.

If you’d like to figure out how much of a credentialist or non-credentialist you are, you can take our credentialist test here.

This piece was first written on February 28, 2024, and first appeared on my website on March 22, 2024.

(1) Two groups have a small (but meaningful) difference in their average value of some trait, with heavily overlapping distributions.

(2) Some people (“Dichotomizers” or “Oversimplifiers”) observe this difference (in their everyday life or media reports) and turn this small average difference into a (sometimes very harmful) oversimplification: “A’s are like this, B’s are like that.” They also fairly often make it seem like this difference is large (or applies to almost everyone in the group), important, and fundamental (e.g., inherent and unchangeable).

(3) Other people (“Difference Deniers”), often acting with good intentions, criticize this oversimplification, which they correctly perceive as harmful. But instead of saying some combination of:

• “the difference in averages is small”
• “the distributions are heavily overlapping”
• “judging an individual based on a small difference in group averages is a poor way to make predictions, as well as unjust”
• “we shouldn’t judge people for differing on that trait” (if it’s not a trait one should be judged on)
• “if we want to remove the difference in averages, we should consider implementing policy XYZ”

they say “the difference in averages does not exist.” After denying the difference and seeing those they respect deny it, some of them become convinced anyone who believes in the existence of this (actually existing) small average difference is nefarious (and lump such people in with those who harmfully oversimplify people into “A’s are like this, B’s are like that.”) Others among them know the average difference exists but pretend not to because they want to fit into the group that adamantly denies the difference, or because they feel guilty about believing it (even though they are right about it existing).

(4) Oversimiplifiers from (2), who remain totally convinced an average difference exists (and are correct about its existence but exaggerate its magnitude), assume that the Difference Deniers from (3) must be either stupid (for not realizing there is a difference), or untrustworthy liars (for denying what they must see is true), or cruel lunatics (for getting enraged at people for believing in “the truth”).

Queue endless fights between the Dichotomizers and the Difference Deniers, both of which are misrepresenting the actual reality of the situation and demonizing each other.

The problem with turning small averages into “A’s are like this, B’s are like that” is that it is an inaccurate oversimplification and often unfair to A’s or B’s or both.

The problem with denying the existence of average differences that, while small, really do exist is that you end up believing falsehoods, or you end up lying, or both, and you may end up unfairly misjudging people who are (without malice) reporting on real average differences.

To avoid the weaknesses of both the Dichotomizers and the Difference Deniers, I think the best way to handle these cases is to:

1) Avoid pre-judging people based on their membership in broad groups – learn about people as individuals before coming to judgments about them.

2) Avoid language like “A’s are like this, B’s are like that” so that you aren’t a Dichotomizer.

3) Avoid denying that an average difference exists when it really does exist, so that way, you aren’t a Difference Denier.

4) When relevant, remind people that small average differences are not a good basis for judging individuals (epistemically and morally), and point out that the distributions between the two groups are heavily overlapping (when they are) to combat people using differences in the average as a justification for stereotyping.

5) Point to (when relevant, helpful, and accurate) policies that may help close the gap between the two groups (keeping in mind that some gaps in averages are fine if the trait in question is merely a difference and not something “good” or “bad”)

6) Point out (when the difference in question is not something people should be judged for) that this attribute should not be a basis for judging people, i.e., that having different values of that trait is completely okay.

Another approach that can be taken when the group differences in the average are small but meaningful is well described by Guy Srinivasan in the comments on an earlier draft of this post: “Can we agree to make decisions as if there were no average difference, since usually all such decisions would turn out the same, and usually when they wouldn’t it’s perpetuating systemic problems to make the decision differently?”

Of course, as with any binary categories, some people will only be partial Difference Deniers or Dichotomizers – people are absurdly complex, and this model I present here is purposely simplified in order to help communicate this dynamic clearly.

Okay, but are there cases where the Dichotomizers or Difference Deniers are actually just right?

Absolutely, there are some.

When a group difference is SO huge that the distributions are nearly non-overlapping, then it’s reasonable to say, “A’s are like this, and B’s are like that.” For instance, it makes sense to say that “blue whales are big, mice are small.” In such cases, the Dichotomizers aren’t really oversimplifying. But when we’re talking about human groups, this kind of situation is very rare.

And in situations when the difference in averages between groups is so small as to be essentially insignificant for all purposes, the Difference Deniers aren’t actually denying reality. For instance, if it turns out that right-handed people are 0.001% better at school than left-handed people, that difference is so small as to not be meaningfully different from zero for all purposes, and so saying there is “no difference” is an extremely reasonable thing to do. There are, in fact, many attributes along which human groups differ so little that “no difference” is an accurate way to describe it (even though the difference is not literally zero to the 10th decimal point).

(1) Linus Pauling won TWO Nobel prizes – one in peace and one in chemistry. Unfortunately, he eventually became obsessed with and widely promoted the false (and sometimes still repeated) idea that high-dose vitamin C cures many diseases, including HIV and snakebites.

(2) Isaac Newton, who co-invented calculus and discovered the laws of gravity, also was convinced the Bible had hidden messages he could decode for prophetic purposes, and spent a lot of time trying to create the mythical philosopher’s stone, so he could turn metal into gold.

(3) Alan Turing, often considered to be the father of theoretical computer science and artificial intelligence, seemingly was convinced by the existence of extrasensory perception. He wrote: “the statistical evidence, at least for telepathy, is overwhelming.”

(4) C. Langan, who appears to have one of the highest IQs ever recorded, believes “you can prove the existence of God, the soul, and an afterlife, using mathematics.” and has claimed that 9/11 was an inside job staged by the Bush administration.

My point is not that these people were stupid – they are the opposite of stupid – they are far smarter than 99.9% of the people that have ever lived (by at least some reasonably common ways of thinking about intelligence). My point is that even the smartest among us hold some silly, false beliefs – intelligence is not enough to avoid error.

Rationality (in the sense of evaluating evidence in such a way as to effectively arrive at the truth on important topics) and intelligence, while related, are also not the same things. Rationality involves actively working to disprove your own beliefs – which intelligent people may or may not do. For instance, intelligence is often used to come up with clever and convincing arguments for why what you already think must indeed be correct. In other words, intelligence can be deployed for rationality but also for rationalization.

Of course, it may also be me that’s wrong. Perhaps there’s a philosopher’s stone, vitamin C cures a ton of diseases, 9/11 was an inside job. But more likely, I’m wrong about other things (that I have no clue I’m wrong about). It’s useful to remember: we all believe false things.

This piece was first written on July 16, 2023, and first appeared on this site on August 16, 2023.

That peak reflects the total sum of your achievements according to your intrinsic values. This may include, for instance, your happiness, the happiness of your loved ones, your positive impact on the world, living virtuously, achieving your deeply meaningful goals, and so on.

Unfortunately, the mountains you face are foggy as hell. Plus, they have dense forests, huge boulders, and brambles covering them. Your mountains are untamed, uncharted.

• The fog means that you can only see clearly for a short distance, and the further you look, the harder it is to tell what’s out there.
• The dense forests mean that to go a considerable distance in most directions, you’ll have to whack your way through with substantial effort.
• The huge boulders will sometimes make a path impassible that had looked promising from around the bend.
• The brambles mean that certain paths will cause considerable pain if you take them. Even more inconveniently, beautiful grasses and flowers sometimes conceal the brambles.

These are your personal mountains, unique in all the world. Your mountains are determined by a combination of:

(1) Your intrinsic values. It is your values that determine the height of each landing and peak, including the height of wherever you’re standing right now.
(2) The structure of the real world, which makes some paths easier to traverse than others. The locations of the forests, boulders, and brambles are metaphors for this structure.
(3) Your current life situation. This is represented by your current latitude and longitude on the mountain range, as well as your physical and mental health, resources, and skills.

Looking out from a distance, you can see the dim outlines of many high-up peaks far away that look promising, but they are in different directions from each other. That means you’ll have to make tough choices about what direction to go, even at the beginning of your journey.

This journey will take your entire life. If you’re like most people, it will be long and hard but also too short. It will be wondrous, terrifying, joyful, and sad.

Rather than trying to travel a great distance in order to climb to great heights, it is easier to find the first comfortable spot, set up a hammock and tent, and make camp there forever. Who can blame you for making that choice (except, perhaps, yourself)?

If you decide to take the journey, you’ll need to use your tool belt, which (if you’re lucky) comes equipped with five (metaphorical) tools.

To have the greatest chance of reaching the highest peaks, you’ll want to train yourself to be a master of each tool.

Here are the tools to master.

Tool 1: The walking stick, which is what you use to move forward along the path you’ve chosen.

By far, the most common tool you’ll use is the walking stick. For every choice of path, you’re going to have to spend a lot of time walking.

Maybe you’ll give up and turn back at the first encounter with a snake, tiger, or tornado. Or maybe you’ll use the walking stick to keep going.

You’re using the walking stick when you create a to-do list and tick items off of it. You’re using it when you push through fear to do something valuable.

Mostly, climbing a mountain involves using your walking stick, but if you only use that, you’re doomed.

Tool 2: The telescope, which allows you to peer at the shape of the mountain, collecting data and facts about the world that you can use to select your path.

The questions that guide your use of the telescope are ones like:

• What questions could I ask that would help me choose my path?
• What do I need to know about the mountains that I don’t yet know?
• What important question am I confused about?
• Where is my lack of knowledge showing?

The scope, like each of your tools, takes many forms. In a start-up, it may look like talking to customers, running surveys, examining other products, or scrutinizing the structure of your own product, keeping a keen eye out for flaws. In your career, it may look like researching career paths, talking to others who have tried different routes, quickly trying things out, and soliciting feedback on your work from your colleagues.

What you’re looking for with the telescope are indications of which nearby paths lead quickest up the mountain, as well as hints for other (potentially faraway) parts of the mountain range that may have yet higher peaks (even if you have to go a ways back down the mountain to get there).

Being good at using the scope means being observant, impartial, curious, methodical, open to criticism, and empirical. And it means being able and willing to cope with reality.

Tool 3: The notepad, which you use to formulate your theories about how the world works, as well as to devise plans.

This tool will most dramatically increase how effectively you use the telescope because there are far too many potential things to point the scope at. Your theories on the notepad, therefore, guide your use of the scope. At the same time, the scope provides data to go into your theories in the notebook.

Some of your theories will be explicit, penned in detail with full awareness, but most will be implicit, born out of the things you’ve seen, etched in your subconscious with a shadowy ink.

In your personal life, your notebook contains your understanding of yourself, your partner, your parents, your friends, and human nature. It contains your understanding of your mountains (as well as other people’s mountains) and your beliefs about where the brambles, boulders, and high peaks lie.

To use the notebook is to sit and reflect, to make predictions, to spell out your thoughts, to reduce ambiguity through precision, to derive new knowledge from other things you already know, to come to new conclusions.

Being good at using the notebook means being thoughtful, philosophical, reflective, logical, cautious, precise, and rational.

Tool 4: The jump rope, which you use to practice and improve your skills.

Sometimes, this will take the form of physical training – maybe you don’t yet have the bushwhacking skills to knock away the brambles. Oftentimes, the training and practice will be mental rather than physical. Maybe you don’t know enough about how to use the scope, or your facility with the notepad is not where it needs to be.

Early on in your journey, you’ll need to use the metaphorical jump rope a lot so that you can build the skills you’ll need for the journey. Over time, you’ll need it less often, but there will always be new skills that are useful to train as you climb higher.

The jump rope is informed by the scope and the notebook. Sometimes the scope will tell you about the sort of paths you’ll soon need to face, and the jump rope will help you prepare for them. Other times, you’ll turn the lens of the scope on yourself to see your weaknesses. You can then use the jump rope to work on these, to reshape yourself.

You’re using the jump rope when you’re reading to learn, taking a course to understand something important, practicing how to do something, asking someone to teach you, breaking a difficult skill into smaller pieces, or asking a question when you’re confused.

If your walking speed is slow, or you find another weakness that is slowing you down, use the jump rope to get that skill up to par. But just as importantly, use the jump rope to hone what you’re already good at, to sharpen it into an exceptional skill.

Tool 5: The whistle, which is how you get the help you need from others.

Most things that are worth doing can’t be done alone, and those that travel without a whistle put themselves in great peril. Sometimes you’ll need the help of others to clear a path, to show you how to use the other tools effectively, or to help you understand why you’re stuck. No matter where you’re headed on the mountains, there are those that have gone that way before who have advice to share.

Using the whistle may mean requesting assistance or a favor, but it can also mean asking advice, asking a simple question, or getting support when you’re mentally exhausted.

Being good at using the whistle means investing time in your relationships, developing deep connections, being a good friend when others use their whistles, meeting new people when you perceive gaps, and being bold enough to call for support when you could use it.

Never travel without a whistle when you don’t have to.

So how, then, do you get to the top of your mountains? Well, you will never get to the very top – the mountains stretch forever. But you can climb high. To maximize your chances, use:

(1) The walking stick (to keep moving forward without giving up).
(2) The telescope (to investigate the mountains carefully and with minimal bias so that you can understand where the brambles and boulders are).
(3) The notepad (to reflect carefully on your beliefs and formulate your plans).
(4) The jump rope (to improve your weaknesses and enhance your strengths).
(5) The whistle (to get help and support).

Good luck – may your climb be a joyous one!

This piece was first written on May 19, 2023, and first appeared on this site on September 22, 2023.

Note: here, I use the term “success” merely in terms of achievement, career success, or high levels of expertise, NOT in terms of happiness, living a good life, morality, having strong social bonds, etc. There is nothing wrong with not wanting to be successful in the way this post focuses on. But if you DO want “success” in the sense in which it is used in this post (or you are interested in being able to predict it in others), you may find some of the models here useful.

I’m also interested to know: which model (below) do you find most useful for thinking about success, and which one of these factors (used in the models) do you think is currently most limiting your success?

1. Noise theory:

success = luck

2. Genetic determinism:

success = (innate) aptitude + luck

Note: whenever I use “luck,” I mean random factors not already accounted for in the other factors in the model. So in the case above, “luck” means luck other than the random chance of what your aptitude is.

3. Traditional right:

success = aptitude + surrounding culture + hard work

4. Social justice left:

success = privilege + luck

5. Economic left:

success = social/economic class you’re born into + luck

6. Cynical theory:

success = some combination of self-promotion, bullshitting, social skills, good-lookingness, starting resources, and luck

7. Gladwell: 

success = whoever practiced for 10,000 hours + luck

8. Dweck: 

success = aptitude + growth mindset + luck

9. Duckworth: 

success = aptitude + growth mindset + grit + luck

10. Seligman: 

success = skill * effort * self-promotion * luck

11. Psychometrics: 

success = IQ + conscientiousness + low neuroticism + luck

12. Ericsson: 

success = luck + hours spent doing “deliberate practice” (i.e., with specific goals and tight performance feedback loops, while analyzing mistakes and dividing skills into micro-skills that can be practiced independently, ideally all done under the supervision of expert coaches)

13. My mega model:

success at a fixed goal = luck^a

* (resources+opportunities)^b

* (community/collaborator quality and supportiveness)^c

* (innate aptitude at relevant skills)^d

* intelligence^e

* rationality^f

* (creativity and resourcefulness)^g

* (social skills)^h

* (hours of deliberate practice)^i

* (unitary or obsessive focus on the goal)^j

* (conscientiousness and self-control)^k

* (physical or mental health)^l

* confidence^m

* (ambition and agency/self-directedness)^n

* (self-promotion skill and effort)^o

* courage^p

* (goal/task-specific factors)^q

* (efficiency and prioritization)^r

Each exponent a, b, c, …, r is a different number from 0 to 1. Note that each of these traits is selected because I believe, on average, having more of them improves the chance of success – that’s why I exclude negative exponents. Furthermore, I’m claiming that these factors, on average, each have diminishing marginal returns. That’s why the exponents are each less than 1 (making a concave function).

The values of the exponents vary depending on the field and type of skill. For instance, in some areas, courage is a minor factor (in which case the courage exponent, n, would be close to 0, and in other fields, courage is essential, in which case n would be close to 1). So, in other words: success is a PRODUCT of roughly 18 factors, and how much each factor matters depends on what you’re trying to do.

Note that this is designed so that if you have literally 0 of any factor, then the level of success is automatically 0 (since 0 times any number is 0). For instance, if you have literally no physical health, you are, presumably, dead, and if you have literally no ambition, presumably you just sit around all day or do the minimum you need to eat.

It’s worth noting that the factors above are not completely statistically or causally independent in reality (becoming higher in one may make you higher in another, on average). But I think the enormous extra complexity of trying to account for these dependencies probably is not worth it in practice.

How do you improve your odds of success?

A lot of times, when people are extremely successful, I think it’s because they avoid being TOO low in any of the factors, and they have one or two factors where they are exceptionally high. Many factors are “bounded” ones: for instance, you can’t work more than 24 hours per day. So it’s impossible to work more than 3x the amount the average person does. But there are some “unbounded” factors where you can potentially be WAY higher than the average person (e.g., “creativity”), which can drive the success score very high (as long as no other factor is close enough to zero to drag it back down). Hence, this model leads to an approach for thinking about how to be more successful (if that’s something you care about).

Put simply, success often flows from not being TOO weak on really important factors and having one or two really strong (and relevant) strengths.

Getting into more detail, here is a process you might use to consider how to increase your odds of great success:

1. For the goal/task you’re trying to succeed at, figure out which of the above factors matter substantially (which maps onto trying to – very roughly – figure out the exponents for each factor).

2. If your strong/weak factors are not a good fit for the goal, consider changing the goal to better play to your strengths, or consider teaming up with someone (e.g., a co-founder) to compensate for your weaknesses.

3. Once you have settled on a goal, identify any especially low factors (relevant to that goal) that are driving your potential for success down, and think about how you can improve at those. Due to multiplicative effects, very low factors can really drag down your potential for success. For instance, if you have severe mental health challenges that interfere with your day-to-day tasks, working on that first can be a great idea (even if you’re just optimizing for success).

4. Identify your strongest factors (that are relevant to that goal) and think about how you might improve at them or hone them to get them VERY high. You can also figure out how to make even more use of these great strengths of yours to achieve good outcomes. Often, one of the most effective things we can focus on is leaning into our greatest strengths (for instance, by designing a path towards our goals that leverages them or working to enhance them even more). This is especially the case once we’ve gotten barriers to success out of the way (i.e., we’ve worked on improving our especially low factors).

A question for you: right now, which of the factors listed above is the one that is most significantly limiting your success?

This piece was first written on September 12, 2021 and first appeared on this site on March 25, 2022.

1. Mimicry: when our in-group promotes falsity that we copy

2. Incentives: when we predict that knowing the truth would feel bad or harm our objectives

3. Complexity: when the truth is hard to figure out

Examples:

1. Mimicry

• Some are Christians because all their friends and family are, too; some are atheists for the same reason.

• Some think that it makes sense to circumcise baby boys because the people they know think it’s healthy and normal; others think it’s bizarre because the people they know think foreskin is healthy and normal.

• Some believe it would harm Black Americans to defund police because their friends say so; others think it would help Black Americans because their friends say it.

2. Incentives

• If you make more money believing X, it’s going to be harder to stop believing it.

• If the idea of permanent death is terrifying to you, it’s going to be harder to stop believing in reincarnation.

• If it would make you feel really bad to find out you were wrong about something you posted online, your immediate reaction may be to deny being wrong (to others and to yourself) to shield yourself from the negative feelings.

Note that mimicry and incentives can blend together. Sometimes we mimic to fit in or to avoid being socially punished. But mimicry is even more basic than that: we seem to have a natural, in-built strong tendency to copy others. If all the people around us believe something, we usually will come to believe it too, without even questioning whether it might be false or even being aware that we copied the belief from others. 

If we see others all behave in a certain way, we’ll probably behave that way, too, unless we have strong reasons not to. This appears to be an evolutionary survival mechanism – it’s risky to (for example) eat plants that are different than the ones your tribe eats (they might be poisonous) or to avoid the behaviors everyone else does (those behaviors might be key to survival in some hidden way). In the wilderness, you can’t figure out how to survive from first principles (chances are you’ll die way too fast for that) – you need to mimic what has worked for centuries (some of which will be key to survival, some of which will be pointless, but evolutionary pressures will have weeded out most of the really harmful stuff and hung on to the most useful stuff).

3. Complexity

• It’s really not obvious how to prevent future risks from advanced artificial intelligence (though it often seems obvious to folks who’ve spent almost no time thinking about it).

• How best to prevent economic crashes is a fundamentally complicated question.

• Nobody actually seems to know how to cure cancers in general.

A weird thing about these three causes for believing falsehoods is that we are usually not aware of their effects.

1. We don’t usually realize it when we believe something just because we copied our social group.

2. We don’t usually realize when we believe something just because it would hurt us not to believe it.

3. We don’t usually realize when our analysis of a complex issue is oversimplified and misses important considerations.

Believing in falsehoods feels just like believing the truth – until the moment we genuinely face up to the possibility of being wrong.

So how do we be right more often?

1. To combat mimicry: we can keep our identities smaller (or have more of them), be more willing to be viewed as having “weird” beliefs, join social groups that value diversity of thought, learn to do less social mimicry (e.g., having greater skepticism towards in-group consensus). We can recognize that every in-group will get some things wrong (including ours) and that it’s worth investigating where ours is wrong. In summary, we can combat mimicry with social resilience and skepticism.

2. To combat harmful incentives: we can recognize that, while there can be short-term pain from accepting the truth, being truth-seeking is usually a better long-term strategy (especially because you can’t just suddenly decide to be truth-seeking when it’s convenient – it’s best to have a habit of being truth-seeking all the time). We can consider thought experiments like: “If X were true, would I rather believe it or be wrong about it?”. We can also leave “lines of retreat” so that we can decide what we’d do and how we’d move forward if we turn out to be wrong on something important. In summary, we can combat bad incentives with a Scout Mindset and a focus on seeking the truth.

3. To tackle complexity: we can use probabilistic thinking, consider multiple hypotheses, and consider the evidence for and against each one. We can train ourselves in evidence and argument evaluation, practice steel-manning arguments, talk to smart and knowledgeable people with different views and fight back against overconfidence (e.g., through calibration practice). We can also do large amounts of research when it’s important to be right. In summary, we can combat complexity with good epistemic hygiene, honed thinking skills, and self-skepticism.

This piece was first written on June 3, 2021, and first appeared on this site on November 11, 2022.

People often talk about what a problem “confirmation bias” is. But we rarely discuss what causes so many of us to search for information in a biased way.

Let’s explore some of the forces:

1. Echo chambers: our routine sources of information tend to support our worldview. Much of this is due to social ties (we tend to talk to people who are similar to us in age, geography, religion, etc.) We also trust news sources more if they share our basic ideology/assumptions. The authorities we look to will be the ones that agree with us on most of our fundamental assumptions (even if some of these assumptions could turn out to be wrong).

2. Soldier Mindset: as Julia Galef explains in her wonderful new book (the Scout Mindset), we are often not even TRYING to figure out the truth. We’re just trying to beat the other side or prove a point. In these cases, of course we have a biased search process.

3. Lack of doubt: when we’re really confident our basic premises are correct, we don’t see the need for a nuanced information search process. We’re going to go to whatever sources are convenient for filling in minor details, even if our beliefs have major unquestioned assumptions.

4. Advocacy: others are actively trying to get us to believe certain falsehoods. They do so through ads, websites, news, and other channels. For the most part, they themselves believe these falsehoods (promoting “the truth” as they see it), but occasionally it’s pure manipulation.

5. Fear of being wrong: it hurts to be wrong, especially if we’ve made the error publicly, have our identity tied up in the belief, or it challenges our understanding of the world or who to trust. We sometimes avoid finding out we are wrong the way we avoid touching a hot stove.

We don’t always seek out information in a biased way. For instance, when looking up driving directions or trying to figure out what paint to use to prevent water damage, we want the right answer, don’t have political biases, and usually have appropriate self-doubt and skepticism.

But we are liable to have a biased search for the truth when we are incentivized to have particular beliefs, such as when our social world supports just one perspective, when we’re trying to prove the other side wrong, when we have no doubt that we are right, when powerful others are devoting a lot of effort to persuade us, or when we’re too afraid of being wrong.

For more about this topic, you may want to check out my recent podcast episode with Julia Galef.

1. Soldier Altruists think it’s obvious how to improve the world and that we just need to execute those obvious steps. They see the barriers to a better world as:

(i) not enough people taking action (e.g., due to ignorance, selfishness, or propaganda), and

(ii) bad groups blocking things (e.g., corrupt politicians or greedy corporations).

2. Scout Altruists think it’s hard to figure out how to improve the world – and most attempts either don’t work, only slightly help, or make things worse. They see the barriers to a better world as:

(i) not enough understanding of causal mechanisms (e.g., due to a lack of high-quality evidence, not enough attention to the evidence we do have, not enough careful reasoning, etc.), and

(ii) too much investment in bad solutions (e.g., due to people jumping to conclusions, doing what feels good emotionally rather than what is effective, ideological blindspots, inertia, etc.)

Soldier Altruists say we need to DO and FIGHT more. Scout Altruists say we need to THINK and TEST more. Soldier Altruists are more likely to think that if we could just get people to be less selfish and more motivated to act, we would make a lot of progress towards a better world. Scout Altruists are more likely to think that if we could just get people to pay more attention to evidence and to have more good-faith debates with strong norms around the quality of argumentation, we would make a lot more progress.

Soldier Altruists may think Scout Altruists are far too reluctant to act and are wasting their time on research and debate. Scout Altruists may think Soldier Altruists are far too confident in their conclusions and are wasting their effort pushing for changes that aren’t going to help much (and which, in some cases, might even make things worse). Of course, in reality, there is a continuum between these two positions. So, on a scale from 0 (Soldier Altruist) to 10 (Scout Altruist) where do you fall? I’m probably a 7 or 8.

As some commenters have pointed out, there is a relationship between this distinction and “Conflict Theory” vs. “Mistake Theory.” I think it is related – but also distinct in important ways. Conflict theory says that there is a giant zero-sum struggle (groups fighting over fixed resources). Whereas in this case, we’re operating from a framework of altruism: “the world can be made a lot better – what’s the big barrier to that happening? Is it that we know what to do and we’re not doing it enough/with enough energy, or is it that we don’t really know what to do?”

Also, to clarify another important point brought up in the comments: I’m not asking, “do you think it’s obvious how we should improve the world if we had a magic wand that could change whatever we wanted?” – instead, the question is: “is it obvious what to do to improve the real world, given that we don’t have a magic wand?” Do we just need to put more money/time/effort/people into executing the current “obvious” strategies because they will work well if we just scale them up? Or is it pretty unclear what strategies we should even be putting more resources into (meaning that a lot of thinking, research, debate and/or evidence evaluation will typically be necessary to even figure out what is worth scaling up)?

Julia’s book (which I highly recommend): https://www.amazon.com/Scout-Mindset-Perils-Defensive-Thinking/dp/0735217556/ref=nodl_

This piece was first written on April 23rd, 2021, and first appeared on this site on January 7th, 2022.

If you liked this piece, you may also like 50 “Laws” of Everything.

This piece was first written on March 21, 2021, and first appeared on this site on March 18, 2022.