What I Learned From One Of The Most Effortful Studies On Behavior Change Ever Conducted

January 13, 2023

I’d like to tell you about one of the biggest and most effortful studies on behavior change ever conducted and its strange findings (that may not even be obvious if you read the paper – I only realized its implications on my 2nd reading!) Let’s begin: The study had 61,000 participants, all with gym memberships. 30 scientists worked in small teams to develop interventions, each aiming to increase gym attendance over 4-weeks. This resulted in 53 interventions to test. A promising setup to figu...

Demystifying p-values

December 31, 2022

There is a tremendous amount of confusion around what a p-value actually is, despite their widespread use in science. Here is my attempt to explain the concept of p-values concisely and clearly (including why they are useful and what often goes wrong with them). — What's a p-value? — If you run a study, then (all else equal, aside from rare edge cases) the lower the p-value, the lower the chance that your results are due to random chance or luck. More precisely: a p-value is the probab...

14 Signs A Study Is More Likely To Be Trustworthy

July 2, 2020

Science is incredible, but not all studies are reliable. In fact, they frequently aren't. So how can we tell when a study is trustworthy? Here are 14 signs to look for (for studies involving humans): 1) Large: a reasonably large number of study participants (20 per group is usually too small - the sample size should be large enough to give the study the power it needs to have a good chance of detecting an effect if the effect is indeed real) 2) Controlled: if the study is making claims to...