Three reasons to be cautious when reading data-driven “explanations”

September 10, 2023

Photo by Sunder Muthukumaran on Unsplash

Did you know that fairly often, there will be multiple extremely different stories you can tell about identical data, none of which are false? In other words, the mapping from statistical results to true stories about those results is not unique. This leads to a lot of confusion, and it also implies that claims about "the reason" behind a complex social phenomenon should be interpreted with caution. Here are 3 common situations of this happening, each illustrated with realistic political ...

Did That Treatment Actually Help You?

April 15, 2023

A mistake we all make sometimes is attributing an improvement to whatever we've tried recently. For instance, we may get medicine from a doctor (or go to an acupuncturist) and feel better, so we conclude it worked. But did it actually work, or was it just chance? Here's a trick to help you decide: What matters (evidence-wise) is how likely that level of improvement would have been in that time period if the treatment works relative to how likely that improvement would have been if the treatm...

It can be shockingly hard just to understand three variables

April 19, 2021

In science (and when developing hypotheses more generally), it is very common to come across situations where a variable of interest (let’s call this the dependent variable, “Y”) is strongly correlated with at least two other variables (let’s call them “A” and “B”). Here are some examples: If you’re a psychology researcher investigating possible causes of depression (Y), you may have trouble disentangling the effects of poor sleep quality (A) and anxiety (B), both of which tend to be corre...