(1) Linus Pauling won TWO Nobel prizes – one in peace and one in chemistry. Unfortunately, he eventually became obsessed with and widely promoted the false (and sometimes still repeated) idea that high-dose vitamin C cures many diseases, including HIV and snakebites.

(2) Isaac Newton, who co-invented calculus and discovered the laws of gravity, also was convinced the Bible had hidden messages he could decode for prophetic purposes, and spent a lot of time trying to create the mythical philosopher’s stone, so he could turn metal into gold.

(3) Alan Turing, often considered to be the father of theoretical computer science and artificial intelligence, seemingly was convinced by the existence of extrasensory perception. He wrote: “the statistical evidence, at least for telepathy, is overwhelming.”

(4) C. Langan, who appears to have one of the highest IQs ever recorded, believes “you can prove the existence of God, the soul, and an afterlife, using mathematics.” and has claimed that 9/11 was an inside job staged by the Bush administration.

My point is not that these people were stupid – they are the opposite of stupid – they are far smarter than 99.9% of the people that have ever lived (by at least some reasonably common ways of thinking about intelligence). My point is that even the smartest among us hold some silly, false beliefs – intelligence is not enough to avoid error.

Rationality (in the sense of evaluating evidence in such a way as to effectively arrive at the truth on important topics) and intelligence, while related, are also not the same things. Rationality involves actively working to disprove your own beliefs – which intelligent people may or may not do. For instance, intelligence is often used to come up with clever and convincing arguments for why what you already think must indeed be correct. In other words, intelligence can be deployed for rationality but also for rationalization.

Of course, it may also be me that’s wrong. Perhaps there’s a philosopher’s stone, vitamin C cures a ton of diseases, 9/11 was an inside job. But more likely, I’m wrong about other things (that I have no clue I’m wrong about). It’s useful to remember: we all believe false things.

This piece was first written on July 16, 2023, and first appeared on this site on August 16, 2023.

What matters (evidence-wise) is how likely that level of improvement would have been in that time period if the treatment works relative to how likely that improvement would have been if the treatment is useless.

For something like tiredness, which tends to fluctuate a lot, feeling somewhat less tired than normal after two weeks may provide almost no evidence a treatment worked. But if you feel less tired than you have in 10 years, that could be strong evidence!

To give another example, if you’ve had a rash without a break for years, and the rash goes away in one day with a new cream, that is very strong evidence the cream worked. But if the rash very often comes and goes on its own, or it took six months of using the cream before it disappeared, its disappearance provides little evidence of effectiveness.

More formally, the amount of evidence an improvement gives you (in favor of the treatment working) is:

Bayes Factor = the probability that you’d see this level of improvement given that the treatment works / the probability that you’d see this level of improvement given that the treatment doesn’t work

In words, this is just “how many times more likely is it that you’d see this level of improvement during this period of time if the treatment works compared to if it doesn’t work.”

This Bayes Factor is what you multiply your prior odds by. So if, before trying the treatment, you thought there were 1 to 3 odds of it working (i.e., a 25% chance), and if you now you get a Bayes factor of 6, you should now believe there are 6*(1/3) = 2 to 1 odds that it works (i.e., a 66% chance).

While it’s rare to be able to do this calculation precisely, it’s this general way of thinking (in terms of relative likelihoods, comparing a world where the treatment works to one where it doesn’t) that’s important. I find this to be an especially helpful application of Bayes’ rule which can guide practical decision-making (e.g., whether to stick with a new treatment).

This piece was first written on April 15, 2023, and first appeared on this site on August 2, 2023.