The reason stems from my thinking that all of the following are important and true (and too often overlooked) regarding science:

1) A lot of science is real AND valuable to society.

2) A lot of “science” is actually fake – see, for instance, a decent percentage of papers in psychology 15 years ago.

3) “Science” (as an approach to knowledge discovery) is one of humanity’s greatest inventions – but in practice, it is reasonably often misapplied, or the process is distorted due to bad incentives or poor training. Unfortunately, not all fields of science have done a good job of being self-correcting either, so sometimes, fields go in bad directions for quite a while and need reform. There are different kinds of bad science:

(i) Sometimes, science is “bad” because it uses unsound methods for figuring out the truth (such as when p-hacking is rampant).

(ii) Sometimes it is “bad” because it overclaims (e.g., “Importance Hacking” where scientists claim they found something important/valuable when they didn’t actually demonstrate what they claim in their study. Or cases where science is used to “prove” questions that can’t be proven by science – such as which policy is better in a particular context when it’s actually a tradeoff between different values).

(iii) Other times science is bad because it is biased (e.g., when people are only willing to run or publish studies that show X but not that show the opposite of X).

(iv) And sometimes science is bad because it’s simply fraudulent.

4) Promoting broad “trust the science” is misguided (and actually harmful) because a bunch of science is fake. If you tell people to always just “trust the science,” then you are going to cause them to be tricked by a bunch of bad science, or you are going to contribute to their disillusionment and loss of trust when they discover (correctly) that some of the science you’re saying is good is actually garbage.

5) The “distrust all science” view is probably an even worse take than “trust the science.” If you distrust all science, you are likely to miss out on incredible things (such as highly effective treatments), and you set yourself up to fall for tons of things that don’t work (e.g., widely used unscientific treatments). Those who tell people to always just “trust the science” sometimes accidentally push people into the “distrust all science” view when those people realize that some of what they are being told to trust is crap.

6) So, hard as it is, rather than promoting either “trust all science” or “distrust all science,” the course of action I believe in with regard to science education is to teach people that “Science” (as a method) is an incredibly powerful and useful invention, but that “science” (as actually practiced) is much like every other field: some of it is good, some of it is crap. There are good hairdressers and bad hairdressers, and there is good science and bad science (and unfortunately, some bad science ends up in the very top journals – while journals and peer review absolutely do block some bad science, they unfortunately still let through quite a lot of it).

Since some science is well done, and some of it is poorly done, it’s very valuable to learn to tell the difference to make the best use of scientific results – both with regard to applying it in your own life and using it to form your beliefs about the world.

If we pretend science is all good or all bad, we do a lot of harm. We need nuance to see through the bad stuff while maintaining the tremendous benefits.

This piece was first written on November 20, 2024, and first appeared on my website on January 14, 2025.

A significant and pretty common problem I see when reading papers in social science (and psychology in particular) is that they present a fancy analysis but don’t show the results of what we have named the “Simplest Valid Analysis” – which is the simplest possible way of analyzing the data that is still a valid test of the hypothesis in question.

This creates two potentially serious problems that make me less confident in the reported results:

1. Fancy analyses impress people (including reviewers), but they are often harder to interpret than simple analyses. And it’s much less likely the reader really understands the fancy analysis, including its limitations, assumptions, and gotchas. So, the fancy analysis can easily be misinterpreted, and is sometimes even invalid for subtle reasons that reviewers, readers (and perhaps the researchers themselves) don’t realize. As a mathematician, I am deeply unimpressed when someone shows me a complex mathematical method when a simple one would have sufficed, but a lot of people fear or are impressed by fancy math, so complex analyses can be a shield that people hide behind.
2. Fancy analyses typically have more “researcher degrees of freedom.” This means that there is more wiggle room for researchers to choose an analysis that makes the results look the way the researcher would prefer they turn out. These choices can be all too easy to justify for many reasons including confirmation bias, wishful thinking, and a “publish or perish” mentality. In contrast, the Simplest Valid Analysis is often very constrained, with few (if any) choices left to the researcher. This makes it less prone to both unconscious and conscious biases.

When a paper doesn’t include the Simplest Valid Analysis, I think it is wise to downgrade your trust in the result at least a little bit. It doesn’t mean the results are wrong, but it does mean that they are harder to interpret.

I also think it’s fine and even good for researchers to include more sophisticated (valid) analyses and to explain why they believe those are better than the Simplest Valid Analysis, as long as the Simplest Valid Analysis is also included. Fancy methods sometimes are indeed better than simpler ones, but that’s not a good reason to exclude the simpler analysis.

Here are some real-world examples where I’ve seen a fancier analysis used while failing to report the Simplest Valid Analysis:

• Running a linear regression with lots of control variables when there is no need to control for all of these variables (or no need to control for more than one or two of the variables)
• Use of ANOVA with lots of variables when really the hypothesis only requires a simple comparison of two means
• Use of a custom statistical algorithm when a very simple standard algorithm can also test the hypothesis
• Use of fancy machine learning when simple regression algorithms may perform just as well
• Combining lots of tests into one using fancy methods rather than performing each test one at a time in a simple way

The problems that can occur when the results of Simplest Valid Analysis aren’t reported was one of the reasons that we decided to include a Clarity Criterion in our evaluation of studies for Transparent Replications. As part of evaluating a study’s Clarity, if it does not present the results of the Simplest Valid Analysis, we determine what that analysis would be, and pre-register and conduct the Simplest Valid Analysis on both the original data and the new data we collect for the replication. Usually it is fairly easy to determine what the Simplest Valid Analysis would be for a research question, but not always. When there are multiple analyses that could be used as the Simplest Valid Analysis, we select the one that we believe is most likely to be informative, and we select that analysis prior to running analyses on the original data and prior to collecting the replication data.

In my view, while it is very important that a study replicates, replication alone does not guarantee that a study’s results reflect something real in the world. For that to be the case, we also have to be confident that the results obtained are from valid tests of the hypotheses. One way to increase the likelihood of that being the case is to report the results from the Simplest Valid Analysis.

My advice is that, when you’re reading scientific results, look for the Simplest Valid Analysis, and if it’s not there, downgrade your trust in the results at least a little bit. If you’re a researcher, remember to report the Simplest Valid Analysis to help your work be trusted and to help avoid mistakes (I aspire always to do so, though there have likely been times I have forgotten to). And if you’re a peer reviewer or journal editor, ask authors to report the Simplest Valid Analysis in their papers in order to reduce the risk that the results have been misinterpreted.

The only approach I know of that I think has a meaningful shot to help solve such huge problems, which you might call “Scientific Entrepreneurship,” combines two methods into one:

(1) Rigorous science to deeply understand the causal structures of the problem and how strong an effect each cause has (which often will begin with a qualitative approach to understand the outlines of the problem and then move to analysis of carefully conducted measurements).   

(2) A “lean startup” approach, where you try things (guided by your current understanding of the causal relationships), see what happens, and then rapidly course-correct based on the results (and sometimes take large pivots) to iterate towards better and better approaches.

When dealing with these highly complex problems, I think that a “lean startup” approach without rigorous science ends up leading to lots of random attempts that have almost no chance of ultimate success (and sometimes even converge to self-propagating but useless approaches, such as charities that perpetually absorb money without genuinely helping the cause).  

On the other hand, a purely scientific approach without the lean startup iterative mindset often ends up missing critical contextual details that are actually essential for the project to work. You’ll inevitably encounter many specific barriers that the scientific theory doesn’t address. In summary, I think our best bet for solving highly complex world problems is Scientific Entrepreneurship: developing a deep understanding of the high-level causal structures through scientific rigor and then combining that with an entrepreneurial form of rapid iteration.

This piece was first written on May 5, 2024, and first appeared on my website on June 5, 2024.

This piece first appeared on ClearerThinking.org on February 28, 2024, was edited on February 29, 2024, and appeared here with minor edits on March 27, 2024.

Does money buy happiness? Intuitively, the answer is yes: common sense tells us that poverty and hardship make people unhappy. We can use money to buy a lot of things that might make us happier – things like a nicer home, fancier vacations, education for our children, or just the opportunity to have more free time. On the other hand, it’s a cliche that “money can’t buy happiness.” Many admire and aspire to the lifestyles of multi-millionaire celebrities, yet rich and famous people often seem desperately unhappy. 

But what does science say about this question? The answer is relevant for all of us as individuals: how important is it for our happiness to strive to make a high salary? It’s also relevant for states: if policy-makers want citizens to be happy, should they prioritize increasing their wealth, or other things?

So, what’s right? Does money make you happier, or is happiness something money can’t buy? In this article, we tell the tale of scholars’ attempts to find out whether money makes people happier, and why they ended up disagreeing on such an apparently simple question. We think you’ll find the results surprising — particularly the recent saga of how some scientists set out to understand the link between income and happiness.

As usual, we’ll discuss the studies’ methodologies and results in detail throughout this piece. However, if you’re short of time and/or just want to know what are the key takeaways, you jump to the “key takeaways” section, at the end of this article. 

Does money increase life satisfaction?

There’s a clear, cross-cultural relationship between income and ‘life satisfaction’ (or ‘life evaluation’). This is broadly how well someone thinks their life is going, relative to what’s realistic for them. For example, in the Gallup World Poll (a large survey, run in over 160 countries), surveyors measure life satisfaction by asking participants in their native language:

“Please imagine a ladder, with steps numbered from 0 at the bottom to 10 at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time?”

This is known as the ‘Cantril ladder’. Participants with higher incomes tend to place themselves higher on the ladder than those with lower incomes. This is true both across countries and within countries. In poorer countries, people rate their life satisfaction as lower than in richer countries:

Source: Our World in Data

Citizens of poorer countries, where the GDP per capita is less than $5000, have low average life satisfaction (3-5 out of 10), whereas in richer countries like Switzerland, Singapore or the US, the average life satisfaction rating is higher (6 or 7 out of 10). Results are typically similar for other measures of life satisfaction, such as answers to the question “In general, how satisfied are you with your life?”

To complicate things somewhat, the relationship between income and life satisfaction is logarithmic. This means that every time you double someone’s income, their life satisfaction doesn’t double, but increases by a fixed amount (in this case, roughly 1 point on the life satisfaction scale that ranges from 1 to 10). 

Note that the chart above is a “log plot” – you’ll see that on the x-axis, income roughly doubles at every division, going from $1000 to $2000 to $5000 to $10,000. So a straight line on this plot means that every doubling of income is associated with an increase in happiness of a fixed number of points. On a regular (not logarithmic) scale, the graph instead looks like this:

Source: Our World in Data

This means that though an increased income is associated with greater life satisfaction, this is proportionate to how much a person is already making. Let’s say you start making an extra $1000 a year in income: this will impact your happiness a lot if you were originally making $2000 a year, but it won’t make a big difference if you were already making $100,000 a year. To put it another way, for those who are poor, life satisfaction will typically increase quite a bit as they make an additional $10,000, but for those who are rich, this amount of money has far less impact. .

The charts above show the relationship between income and life satisfaction across countries, but higher-income countries and lower-income countries differ in many ways other than wealth. For instance, lower income countries tend to have worse healthcare systems, so it’s hard to tell if they have lower average life satisfaction due to lower incomes or due to worse healthcare systems (or due to a myriad of other differences between wealthy countries and poorer countries). So it’s hard to be confident from this data alone that higher incomes cause greater life satisfaction, rather than some other factors increasing both income and life satisfaction.

That’s why it’s useful to bring in evidence from within countries, to see if the effect holds within a single country as well.

Source: Our World in Data

Each country in the chart above is represented by a line, connecting the average reported life satisfaction of people in five income quintiles within the country. The chart is messy, but in general, the story is the same: most of the lines trend upwards, with richer citizens reporting a higher life satisfaction than poorer citizens.

Source: “Subjective Well-Being and Income: Is There Any Evidence of Satiation?”, Betsey Stevenson and Justin Wolfers, NBER Working Paper 18992, April 2013. Reformatted in the Economist

The chart above shows a more limited sample of countries. This data is older, from 2013, but it shows a similar trend: if we look at the US line, we can see that average life satisfaction ranges from about 6.6 (out of 10) at the lowest incomes, to around 7.5 at the highest incomes. Compare this to the India line: average life satisfaction is lower in India overall, but just as in the US, poorer Indians on average report lower life satisfaction (about 4.2) than richer Indians (about 5.8).

Other ways of measuring life satisfaction also show within-country differences in life satisfaction depending on income. For example, in the chart below, participants in the US were asked whether they were ‘very satisfied’, ‘somewhat satisfied’, ‘somewhat dissatisfied’ or ‘very dissatisfied’ (rather than being asked to rate their satisfaction on a scale). The proportion of ‘very satisfied’ people is higher in higher income bands, and the proportion of dissatisfied people is lower. Although the number of participants is not high in the lowest and highest bands, on the right hand section of the table (Panel B) we see a fairly steady increase in the percent of people who are very satisfied as income rises. 

Source: “Subjective Well-Being and Income: Is There Any Evidence of Satiation?”, Betsey Stevenson and Justin Wolfers, NBER Working Paper 18992, April 2013. 

Note that even this within-country data doesn’t prove that income causes greater life satisfaction: all these polls show is a strong association between income and life satisfaction. Perhaps even within countries, other factors (such as the quality of infrastructure in richer vs poorer areas of a single country) cause people to feel more satisfied with their lives and to earn more money. However, given that across-country and within-country evidence agree, and it’s reasonable to assume a priori that money can contribute to life satisfaction by enabling people to buy things they want and need as well as to have more control over their time, it stands to reason that there is probably a causal link. But we should keep in mind that the data don’t provide absolute proof that an increase in income causes an increase in life satisfaction. 

An additional complicating factor is that this data measures averages. Even if on average it’s the case that greater income causes greater life satisfaction, for any given person, income and life satisfaction could be more or less closely related than average. This means that two people of the exact same income may be impacted very differently by the same increase in income: perhaps the extra money allows one person to leave a career they hate and pursue one they love, whereas the other just puts the additional income into their savings and doesn’t end up using it for anything important. So we have to be careful not to make confident generalizations about individuals, based on the average. 

So, if more money is associated with greater life satisfaction, is it right to say that the wealthier tend to be happier? Not exactly: it depends what you mean by happiness. 

Does more money make you happier day-to-day?

The Gallup polls, and many other studies related to happiness, measure life satisfaction: participants are asked to reflect on their lives holistically and think about how things are going. However, life satisfaction is only one way to measure a person’s happiness. We might also consider how emotionally happy people feel day to day: whether they tend to feel joyous, content and calm, or stressed, depressed and anxious. This is sometimes known as ‘hedonic wellbeing’ or ‘experienced happiness’.

Psychologist Daniel Kahneman and economist Angus Deaton wondered if there was a relationship between this moment-to-moment hedonic wellbeing and money, just as there was with life satisfaction. In a 2010 experiment, they analyzed more Gallup data, this time from 1000 U.S. residents. This survey, rather than asking participants to rank themselves on Cantril’s ladder, instead asked them whether they had experienced specific emotions for a lot of the previous day (for example ‘enjoyment’ ‘sadness’ or ‘worry’).

The researchers then grouped these emotions together into ‘positive affect’ – happiness, laughter and enjoyment – and ‘blue affect’ – sadness and worry. They also measured stress. People with higher incomes were more likely to report experiencing positive emotions the previous day, and less likely to report sadness, worry or stress.

Source: “High income improves evaluation of life but not emotional well-being”, Daniel Kahneman and Angus Deaton, PNAS 107, September 2010

As with life satisfaction, the relationship between hedonic wellbeing and income was logarithmic in Kahneman and Deaton’s data for lower incomes, meaning that doubling a person’s income was associated with a fixed increase in their hedonic wellbeing. However, unlike with life satisfaction, this experiment seemed to show that increases in emotional wellbeing taper off at around $75,000 per year (in the U.S.); after that, extra money didn’t seem to increase wellbeing anymore.

On the graph above, ‘ladder’ refers to Cantril’s ladder, the life satisfaction question asked in other studies. You can see that this line continues to go up while the hedonic wellbeing lines flatten out. Again, note that this x-axis is on a log scale, with income doubling every division.

This provides an interesting contrast to the life satisfaction studies: it seems that if you already make $75,000 a year in the U.S., you might become more satisfied with your life if you made more money, but you’d be unlikely to have more positive emotional experiences day-to-day. This makes some intuitive sense: money helps us meet our basic needs and can solve a lot of problems, but there are other problems that it’s much harder to solve with money and that even billionaires have to face — for example relationship conflict, bereavement, or mental and physical illness. Maybe once you’re making $75,000 a year, you can solve all the problems that are soluble with money, leaving only those trickier problems.

But it turns out that story is wrong! 

Questioning the taper 

It makes intuitive sense that day-to-day, hedonic wellbeing would taper off at higher incomes. However, Kahneman’s and Deaton’s result did not hold up to scrutiny. A decade later in 2021, economist Matthew Killingsworth tried to replicate their conclusion on new data. He found something different: in his analysis, hedonic wellbeing did continue to increase with income, even above $75,000 a year. It didn’t taper off or plateau. The slope of the graph was also the same below $75,000 a year and above $75,000, meaning that on average, a doubling of income from $15,000 to $30,000 a year and a doubling from $75,000 to $150,000 a year are associated with the same increase in emotional wellbeing.

What was going on here? Kahneman (the author of the first paper), Killingsworth (the author of the second paper) and psychologist Barbara Mellers, keen to get to the bottom of this, acted like ideal scientists: they embarked on an adversarial collaboration, all working together to try to work out why Kahneman’s and Killingsworth’s experiments had got such different results.

They discovered a few important differences between the 2010 and 2021 studies. First, the 2021 study used more accurate data collection methods. In Kahneman’s 2010 study, surveyors had asked participants to recall whether they’d felt certain emotions the previous day. But in Killingsworth’s experiment, participants instead received smartphone notifications several times per day that asked them to rate how they were feeling at that moment, on a scale of ‘very bad’ to ‘very good’. Since they asked participants multiple times and asked them to describe their feelings in the moment, rather than relying on memory, they got a more accurate measurement of each participant’s general hedonic well-being. There were also many more data points in the 2021 study: over a million data points from 33,000 people (vs 1000 people in the 2010 study, which when separated into different income buckets makes it a bit thin). Killingsworth’s experiment also surveyed more high-income people and had more different income bands, rather than lumping together all participants who made over $120,000 a year.

However, despite this, Kahneman and Killingsworth (when working together) did find some evidence for a plateau in hedonic wellbeing above $75,000 a year in income. So why didn’t it appear in Killingsworth’s study? Why did Killingsworth find that emotional wellbeing didn’t seem to plateau even at high incomes, up to $500,000 a year and beyond?

When the authors re-examined the data from the 2021 study, they discovered that the plateau did exist — for the 20% of the population with the lowest emotional wellbeing. That is, if you experience lots of negative emotion to begin with, additional income is not associated with increased hedonic wellbeing once you make over $75,000 a year, but if you have average or high hedonic wellbeing, additional income is associated with an increase in day-to-day positive emotions, even if you are already wealthy.

As they investigated, they realized that they had missed this because their methods of measuring wellbeing worked better as measures of unhappiness, or negative wellbeing: they couldn’t easily differentiate between people who were pretty happy, moderately happy, and very happy (in terms of their day-to-day emotions). This made it more difficult to spot that the plateau only existed for the 20% of the population with the lowest day-to-day wellbeing. 

So does that mean day-to-day wellbeing does increase the richer you get?

Well, yes…and no. Media discussions of this story focussed simply on whether happiness increases with income, since that was the main focus of the research. 

But another relevant question — probably more relevant for most people — is how much happier does more money make you? Let’s look at the data (on a regular, non-log scale):

Source: our reanalysis of data from Kahneman, Killingsworth and Mellers’ 2022 adversarial collaboration.

Does it look like average wellbeing increases in the higher income categories? It basically looks like it doesn’t! If you squint, you can just about notice that the bar in the $625,000-a-year band is a tiny bit higher than the bar in the $15,000-a-year band. 

You can only see that logarithmic upwards line if you zoom way in:

Source: our reanalysis of data from Kahneman, Killingsworth and Mellers’ 2022 adversarial collaboration.

Look at the y-axis. In this zoomed-in version, you can see that the lowest point of the curve — the average wellbeing of people making $15,000 a year — is at about 60.9, on a 100-point scale. The highest point — people making $400,000 a year — is 65.8. That is, the difference between the poorest and the wealthiest is only about 5 points, on a 100-point scale!

The takeaway from these hedonic wellbeing studies is often ‘more money makes you happier’. While this isn’t wrong, a more relevant lesson from this research might be that people with vastly different levels of wealth have surprisingly similar levels of emotional wellbeing. 

To put this another way, even if we assume this entire effect is causal (that is, that income is causing all of this increase in emotional well-being), then if you made 25 times more money, taking you from the bottom 10% to well within the top 10% of US incomes, you should only expect to get half a point happier on a 10-point scale (from 6 to 6.5). 

We suspect that many will find the small size of this effect surprising. 

To be fair, the authors of the adversarial collaboration did mention this, but it wasn’t the focus of the paper, and we suspect that many people who looked at the paper briefly would have missed it:

‘[Kahneman and Deaton] reported that the effect of an approximately fourfold difference in income is about equal to the effect of being a caregiver, twice as large as the effect of being married, and less than a third as large as the effect of a headache.’

What is the more interesting finding — that emotional wellbeing goes up logarithmically with income, or that emotional well-being goes up very little with income? We think that the second finding is likely to be the more important one for most people.

Now, of course this result doesn’t mean that you yourself wouldn’t have much higher emotional wellbeing if you were to increase your income substantially — but the link seems weak enough that we should be wary about assuming we’ll feel much happier moment to moment if we make more money. 

Life satisfaction varies much more than hedonic wellbeing 

If hedonic wellbeing varies so little, what about life satisfaction, the variable we discussed at the beginning of this article? The data show that people with greater incomes have higher life satisfaction, on average, but is this difference equally tiny? 

No: if we look again at the within-country data on life satisfaction, we can see that the US line spans more than 1 point on the 10-point scale. What’s more, the difference is even greater if we compare countries: the poorest in India are about 3.5 points less satisfied, on average, than the richest in the US.

Source: “Subjective Well-Being and Income: Is There Any Evidence of Satiation?”, Betsey Stevenson and Justin Wolfers, NBER Working Paper 18992, April 2013. Reformatted in the Economist

So, life satisfaction, in general, seems to vary much more with income than hedonic wellbeing. 

–

To make a more direct comparison, looking at US data, every doubling in income is associated with an increase of 0.6 on 10-point scale in life satisfaction, and an increase of only 0.1 on a 10-point scale in hedonic wellbeing (or equivalently, 6 points on a 100-point scale for life satisfaction, and just 1 point for hedonic wellbeing).

Another way to understand this difference is to measure the variance from the mean of each measurement, which Killingsworth did in his 2020 study. 

Source: ‘Experienced well-being rises with income, even above $75,000 per year’, Matthew Killingsworth

Note that the x-axis here is logarithmic (with incomes doubling at each division), and the y-axis uses not absolute scores but z-scores, which measure how many standard deviations each participant’s score is from the mean. The blue “life satisfaction” line slopes up more sharply than the red hedonic wellbeing line, meaning that as income goes up, life satisfaction rises faster than experienced well-being (it increases by a greater number of standard deviations).   

This is counter-intuitive: if people’s day-to-day feelings are middling (as they seem to be, on average), shouldn’t this lead to a middling level of life satisfaction? Are people bad at remembering their day-to-day emotions, so that when they answer life satisfaction questions, they overestimate or underestimate how good their life is?Perhaps. But more plausibly, life satisfaction and hedonic wellbeing are different elements of happiness, rather than two different ways to measure a single, unitary trait. Most people value positive emotions, but they also value other things, including career, family, status, material goods, achieving their goals, and money itself. And sometimes, striving to achieve something you aspire to — something that would increase your life satisfaction — will cause more negative emotions in the short run — for example, when you overwork yourself and take on lots of stress to hit a work milestone. 

Key takeaways – Are people, on average, happier when they make more money? 

We’ve learned that:

• Making more money is associated with moderately greater life satisfaction 
• Making more money is associated with a very small increase in hedonic wellbeing 
• …unless we’re talking about is the unhappiest 20% of people who already make $75,000 a year or more in the U.S., in which case additional income was not associated with greater hedonic wellbeing
• In both cases (life satisfaction and hedonic wellbeing), extra income has a greater effect when you start off with less money – or, more precisely, each doubling of income is associated with the same effect.  

So, the link between income and happiness  depends on what you mean by happiness, and how much money we’re talking about. Ultimately, you can make some predictions about the way you might feel if you had more or less money, but like so many scientific questions, the answer is messy, and you’re unlikely to know for certain how things will turn out. Perhaps this is why our cultural clichés give us such mixed signals about this question! 

Building Happiness Habits

You’ve learned how income is related to happiness, but if you’re interested in exploring habits that could help you improve your day-to-day hedonic wellbeing, we encourage you to test Clearer Thinking’s “Building Happiness Habits” interactive tool.

It works by pairing certain happiness techniques – namely mindfulness and gratitude – with everyday activity triggers such as walking or checking social media, thereby cultivating happiness-improving mental habits. 

We tested each technique on hundreds of users and found that they made a positive difference for those who practiced them, in just 3 days.

Each technique is:

• Easy to learn – there’s no need for long, difficult training in order to be able to perform them.
• Simple to apply — there’s just one main thing to do, not a bunch of steps.
• Time-efficient — it takes just a few minutes a day to practice them.

Ready to make your days a little brighter?

But part of what makes me an extreme non-credentialist is that if I spend an hour watching someone with a nutrition Ph.D. debate a completely self-taught person, and the Ph.D. is making bad arguments and pointing to weak evidence, and the self-taught person is making very solid arguments and pointing to strong evidence and has a very solid command of the relevant facts, the fact that the first person has a Ph.D. will be nearly completely washed out for me at that point, and I will trust the second person’s view of nutrition far more based on the quality of their thinking and the reasons underlying why they believe what they do.

So, being a non-credentialist to me isn’t about thinking that credentials are meaningless, but rather, it involves being willing to quickly update away from the evidence of a credential once you have more direct evidence about the way a person comes to conclusions and what they know.

Most Ph.D.s in a subject are vastly more reliable sources of information on that subject than most non-Ph.D.s on that same subject, but there are lots of exceptions, and sometimes self-taught people are absolutely world-class (and, in any human endeavor, plenty of people with fancy credentials are actually full of B.S.)

Another thing that makes me a non-credentialist is that I love to see highly credible, highly knowledgeable, self-taught people discussing topics and spreading their ideas (whereas some people are very much rubbed the wrong way when someone is talking publicly about a topic they lack a credential in).

An important note: when there is a strong scientific consensus, that is usually a strong starting point for beliefs on topics you know little about (e.g., in physics or biology), even though the consensus is not always right. But trusting the scientific consensus is not the same as trusting one person due to their credentials – a strong scientific consensus is typically more reliable than individual experts.

If you’d like to figure out how much of a credentialist or non-credentialist you are, you can take our credentialist test here.

This piece was first written on February 28, 2024, and first appeared on my website on March 22, 2024.

According to YouGov polling, 41% of people in the United States think that it is the greatest country in the world. Others see the U.S. as a place full of arrogance, violence, and inequality. So, what’s the truth? 

The truth is that there isn’t a single notion of what makes something the “best.” To explore how great (or not) America is, we’ll start by looking at the question from multiple angles. We’ll see how the U.S. stacks up according to a number of important factors before we decide how great it really is:

 1. Technology 

The U.S. is among the best countries in the world for technology and business innovation. It currently ranks 3rd in the United Nations’ Global Innovation Index. It has the most startups worth over $1 billion (the 3rd most startups per capita) and is the originator of many technologies used globally. It is also considered by many to be the best place to create a tech startup. 

2. Violence

The U.S. is a violent place, given its level of wealth, with the 2nd highest number of gun-related deaths in the world and the 2nd highest rate of firearm-related suicides per capita. Among high-income countries with at least 10 million people, it has the highest number of homicides per capita.

️3. Prisons

The U.S. is a country of many prisoners, with the most people in prison of any country in the world and the 6th highest incarceration rate in the world (and the single highest rate among wealthy countries).

Violent crime charges are the number 1 reason people are locked up in the U.S. (though in some cases, the definitions for “violent” can include actions that don’t cause immediate physical harm, such as purse snatching and drug manufacturing). 

4. Wealth

It’s an extremely wealthy place with the highest nominal GDP in the world, as well as the highest GDP per capita (both nominal and PPP) of any country with over 10 million people. California’s economy alone is large enough that, if California were a country, it would rank 6th in the world by some metrics. And by some metrics, Texas’s economy is larger than Russia’s and would rank 8th.

About 9% of U.S. adults are millionaires, and 39% of ALL millionaires are in or from the U.S. 

5. Inequality

The U.S. is a fairly unequal place in terms of wealth and income, with inequality higher than 63% of countries (the Gini coefficient is 39 vs. a world average of 38). It’s the 5th most unequal among the 37 OECD countries, and the average income of the top 20% of earners is 9.4 times the bottom 20% (though note that inequality figures can be dependent on how taxes and social benefits are handled in the calculations).

The wealth gap in the U.S. is especially pronounced across racial lines, with the median Black household having $24,000 in savings vs. the median white household with $189,000 in savings (almost 8x more). 

6. Science

The U.S. is a very scientifically innovative place, with the most Nobel prizes of any country (the 15th highest per capita). It also has the 2nd highest annual patent applications (which is the 4th highest per capita), many of the world’s top universities, and the most clinical trials worldwide. 

7. Health

The U.S. is an unhealthy place relative to its level of wealth. It has the 10th highest prevalence of obesity (the highest of any wealthy country) and the highest rate of death from illicit drugs of any country where such data is known. Life expectancy in the U.S. is 79 years, which places it 47th highest out of 193 countries (so, it’s roughly in the top 25th percentile). 

8. Entertainment

The U.S. is arguably the most influential country in the world in terms of entertainment production, with the most movies made each year of any country, as well as the largest box office sales and the largest music market size. 

9. Healthcare

Many Americans feel let down by U.S. healthcare despite the U.S. spending the most per capita on healthcare of anywhere in the world. Roughly 48% of Americans rate the healthcare system as excellent or good, 31% as fair, and 21% as poor, which are worse ratings than surveys found in the 2010s.

The U.S. has exceptional top hospitals but bad price transparency (so it’s hard to know what you will end up paying) and inflated prices relative to a lot of the rest of the world. 

10. Desirability

The U.S. is rated the single most desirable place to move to for people worldwide looking to emigrate, though its ratings have declined somewhat. In terms of Americans wanting to leave, during the Bush and Obama eras, about 10% of Americans said they’d like to move to another country, and this jumped to 16% in the Trump era. 

11. Military

The U.S. spends a shocking amount on its military, with about 39% of ALL worldwide defense spending being by the U.S. At times its military power has been a stabilizing force worldwide (e.g., against the Nazis). On the other hand, the U.S. has also initiated a number of disastrous wars. 

12. Happiness

The U.S. is a pretty happy place. When Americans are asked to rate their “general satisfaction with life on a scale from 0 to 10”, the average score is a 7 (compared to an OECD average score of 6.7), which places the U.S. 14th highest among 41 OECD countries.

When asked to imagine a ladder, with steps numbered from 0 to 10 at the top, with the top representing the best possible life for you and the bottom the worst possible one,

Americans place themselves at 6.9 on average, which is 16th highest out of 167 countries (i.e., 10th percentile). According to the World Happiness Index, which attempts to evaluate countries’ happiness by combining factors like GPD per capita, social support, healthy life expectancy, freedom to make life choices, generosity, and perceptions of corruption, the U.S. is 16th out of 142 countries (i.e., 11th percentile). ️

13. Ideals

The U.S. has high ideals, some of which are reflected in the Declaration of Independence: “We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.

It sometimes lives up to these (e.g., through its strong personal freedoms), and sometimes has dramatically failed to live up to them (e.g., in its use of slavery and treatment of Indigenous Americans). 

Reaching a verdict: How great is the U.S., really?

So, is the U.S. one of the greatest countries? One approach to this question is to simply say that it can’t be answered because different countries differ in too many ways to make such comparisons possible. Another approach is to say that just one of the factors above trumps all the others.

Our preferred view, however, is to think of it in terms of your values: depending on what your values are, you will weigh the factors above differently. By some sets of values, the U.S. may arguably be the best country, whereas, by others, it doesn’t even come close.

At our organization, Clearer Thinking, we conducted research to figure out what it is that people value intrinsically (that is, what people value for its own sake – not as a means to other things). We’ve organized the results of that research into 22 different categories of common values:

Once you have a list of your own intrinsic values (which you can find out with our Intrinsic Values Test), then you can start to answer the question of how great the U.S. is.

For instance, if you have strong values related to protecting those who are less fortunate, you may give the U.S. lower marks due to its relatively high levels of inequality, whereas if you place more value on achievement, the U.S. may get higher marks due to being highly innovative in technology, business, and science.

It can be easy to base your judgments, including those about how good the U.S. is, on group identity rather than based on careful consideration of the facts and what you value. As political scientists Patrick Miller and Pamela Johnston Conover have said:

“The behavior of partisans resembles that of sports team members acting to preserve the status of their teams rather than thoughtful citizens participating in the political process for the broader good.” 

Employing a framework like the one outlined above, where you:

1. first attempt to impartially consider the facts,
2. then consider your values, 
3. and use the facts to inform how good or bad the thing is according to your values, can help you ensure that you’re deriving your conclusions thoughtfully and carefully, rather than simply deriving them from a desire (conscious or not) to conform to the expectations of group identities.

Ultimately, whether you conclude that the U.S. is great or not, we suggest basing your judgment on an evaluation of the facts plus careful consideration of your own values, not based on other people’s expectations.

This piece was first published on the Clearer Thinking blog on January 3, 2024, and first appeared on my website on January 10, 2024.

(1) Don’t use a net with big holes to catch a small fish

That means you should use a large enough sample size (e.g., number of study participants) to reliably detect whatever effects you’re looking for!

(2) Don’t use calculus to help you assemble IKEA furniture 

That means using and reporting the simplest analysis that is a valid test of your hypothesis (even if you also decide to do fancier analyses). I call this the “Simplest Valid Analysis.” It’s easy to deceive yourself (and others) with overly fancy math!

(3) Don’t claim you saw a bear if all that happened is you heard a growl in the distance

Papers often claim more than they actually show. It’s best not to make such claims OR to point out gaps between what was shown and what was claimed (i.e., other interpretations of the data).

(4) Finding out you’ve backed the wrong horse is better than being a horse’s ass

It feels bad when a theory we’re fond of turns out to be wrong. Even more so when we’ve claimed it in public. But it’s FAR worse defending falsehoods for years because we won’t update on evidence.

(5) When you win at poker, remember that you’re in a casino

Results are sometimes just an artifact of chance. If you tested lots of hypotheses, you should be more skeptical of your own p < 0.05 findings. Don’t forget that all of the averages that you estimate come with confidence intervals.

You can also think about applying these ideas when you’re reading research rather than conducting it. Be more wary of a study when you notice that it has any of these characteristics:

(1) Is small

(2) Overcomplicates things

(3) Overclaims

(4) Is run by people whose incentives don’t align with truth-finding

(5) Runs many tests that fail, and just focuses on a few that don’t, without acknowledging this 

This piece was written on September 22, 2023, and first appeared on this site on October 18, 2023.

This leads to a lot of confusion, and it also implies that claims about “the reason” behind a complex social phenomenon should be interpreted with caution.

Here are 3 common situations of this happening, each illustrated with realistic political examples:

1) Correlated variables
You want to explain why politician X won. If being older and Christian are substantially correlated with each other AND ALSO with voting for X, then you could write either:

• X wins because of older voters!
• X wins because of Christian voters!
  …depending on who the writer prefers to blame it on.

2) Multi-causality
Many events have multiple causes, and so while it isn’t incorrect to say one “caused” that event, it is incomplete and can be misleading.

For instance, suppose you want to explain what caused politician Y to lose in a very close election. Any small deviation would have made the results different. So you could validly say:

• Y loses because fewer Z’s voted than usual!
• Y loses because more W’s voters than usual!

That way, the writer can choose what group they want to try to make into the guilty party. There may be dozens of such small discrepancies that could have “caused” the result.

3) Causal chains

If A caused B, which caused C, which caused D, you could blame D happening on any of A, B, or C, and each story is true (though incomplete).

This can provide convenient excuses to ignore solvable problems. For instance, people often say nuclear power is not adopted much because it’s so expensive. True – but why is it so expensive? If it is regulation that has made it so expensive (which I believe is true in this case), then blaming low usage on cost alone is misleading.

When reading an explanatory story based on data, it’s worth asking yourself: Is this the only story the dataset tells? Fairly often, it won’t be.

This piece was first written on August 7, 2023, and first appeared on this site on September 10, 2023.

This confuses the public and undermines trust in science and scientists.

Some interesting examples:

1) As COVID-19 spread early in the pandemic, epidemiologists confidently stated many true things about it that were scientifically measured (e.g., rate of spread). Some of them were also equally confidently stating things that were just speculation (e.g., its origin being natural).

2) String theorists told the public many true and interesting things about string theory (e.g., why they feel it’s exciting). Some also confidently claimed very uncertain stuff like:”Superstring theory successfully merges general relativity and quantum mechanics.”

Being charitable, perhaps this could be interpreted not as a claim about superstring theory providing a correct theory of physics but rather as a statement about what superstring theory is doing mathematically. Even if so, though, this is – at the very least – going to be very confusing to those who read it. The statement also makes superstring theory seem like it can claim great achievements that perhaps it can’t.

3) Biologists confidently tell the public many true things about how cells form, how evolution works, and so on. Some, unfortunately, have made overconfident claims about a subject that is extremely uncertain: how life formed on Earth. We have only highly speculative theories.

Let me be clear: most scientists don’t engage in what I’m describing above. But when people claim something has been scientifically PROVEN when it actually hasn’t, this tends to reduce trust in the scientific enterprise and causes people to doubt scientists.

My field (psychology) is squishy enough that (unlike physics/biology) little has truly been PROVEN beyond a doubt. At best, we can usually say that studies have found a relationship or that (based on our own interpretation of the evidence) we believe a certain thing.

This piece was first written on August 13 and first appeared on this site on August 23, 2023.

You have probably heard the phrase “replication crisis.” It refers to the grim fact that, in a number of fields of science, when researchers attempt to replicate previously published studies, they fairly often don’t get the same results. The magnitude of the problem depends on the field, but in psychology, it seems that something like 40% of studies in top journals don’t replicate. We’ve been tackling this crisis with our new Transparent Replications project, and this post explains one of our key ideas.

Replication failures are sometimes simply due to bad luck, but more often, they are caused by p-hacking – the use of fishy statistical techniques that lead to statistically significant (but misleading or erroneous) results. As big a problem as p-hacking is, there is another substantial problem in science that gets talked about much less. Although certain subtypes of this problem have been named previously, to my knowledge, the problem itself has no name, so I’m giving it one: “Importance Hacking.”

Academics want to publish in the top journals in their field. To understand Importance Hacking, let’s consider a (slightly oversimplified) list of the three most commonly-discussed ways to get a paper published in top psychology journals:

1. Conduct valuable research – make a genuinely interesting or important discovery, or add something valuable to the state of scientific knowledge. This is, of course, what just about everyone wants to do, but it’s very, very hard!
2. Commit fraud – for instance, by making up your data. Thankfully, very few people are willing to do this because it’s so unethical. So this is by far the least used approach.
3. p-hack – use fishy statistics, HARKing (i.e., hypothesizing after the results are known), selective reporting, using hidden researcher degrees of freedom, etc., in order to get a p<0.05 result that is actually just a false positive. This is a major problem and the focus of the replication crisis. Of course, false positives can also come about without fault, due to bad luck.

But here is a fourth way to get a paper published in a top journal: Importance Hacking.

4. Importance Hack – get a result that is actually not interesting, not important, and not valuable, but write about it in such a way that reviewers are convinced it is interesting, important, and/or valuable, so that it gets published.

For research to be valuable to society (and, in an ideal world, publishable in top journals), it must be true AND interesting (or important, useful, etc.). Researchers sometimes p-hack their results to skirt around the “true” criterion (by generating interesting false positives). On the other hand, Importance Hacking is a method for skirting the “interesting” criterion.

Importance Hacking is related to concepts like hype and overselling, though hype and overselling are far more general. Importance Hacking refers specifically to a phenomenon whereby research with little to no value gets published in top journals due to the use of strategies that lead reviewers to misinterpret the work. On the other hand, hype and overselling are used in many ways in many stages of research (including to make valuable research appear even more valuable).

One way to understand importance hacking is by comparing it to p-hacking. P-hacking refers to a set of bad research practices that enable researchers to publish non-existent effects. In other words, p-hacking misleads paper reviewers into thinking that non-existent effects are real. Importance Hacking, on the other hand, encompasses a different set of bad research practices: those that lead paper reviewers to believe that real (i.e., existent) results that have little to no value actually have substantial value.

This diagram illustrates how I think Importance Hacking interferes with the pipeline of producing valuable research:

There are a number of subtypes of Importance Hacking based on the method used to make a result appear interesting/important/valuable when it’s not. Here is how I subdivide them:

Types of Importance Hacking

1. Hacking Conclusions: make it seem like you showed some interesting thing X but actually show something else (X′) which sounds similar to X but is much less interesting/important. In these cases, researchers do not truly find what they imply they have found. This phenomenon is also closely connected with validity issues.

• Example 1: showing X is true in a simple video game but claiming that X is true in real life.
• Example 2: showing A and B are correlated and claiming that A causes B (when really A and B are probably both caused by some third factor C, which makes the finding much less interesting).
• Example 3: if a researcher claims to be measuring “aggression,” and couches all conclusions in these terms but is actually measuring milliliters of hot sauce that a person puts in someone else’s food. Their result about aggression will be valid only insofar as it is true that this is a valid measure of aggression.
• Example 4: some types of hacking conclusions would fall under the terms “overclaiming” or “overgeneralizing;” Tal Yarkoni has a relevant paper called The Generalizability Crisis.

2. Hacking Novelty: refer to something in a way that makes it seem more novel or unintuitive than it is. Perhaps the result is already well known or is merely what just about everyone’s common sense would already tell them is true. In these cases, researchers really do find what they claim to have found, but what they found is not novel (despite them making it seem so). Hacking Novelty is also connected to the “Jingle-jangle” fallacy – where people can be led to believe two identical concepts are different because they have different names (or, more subtly, because they are operationalized somewhat differently).

• Example 1: showing something that is already well-known but giving it a new name that leads people to think it is something new. The concept of “grit” has received this criticism; some people claim it could turn out to be just another word for conscientiousness (or already known facets of conscientiousness) – though this question does not yet seem to be settled (different sides of this debate can be found in these papers: 1, 2, 3 and 4).
• Example 2: showing that A and B are correlated, which seems surprising given how the constructs are named, but if you were to dig into how A and B were measured, it would be obvious they would be correlated.
• Example 3: showing a common-sense result that almost everyone already would predict but making it seem like it’s not obvious (e.g., by giving it a fancy scientific name).

3. Hacking Usefulness: make a result seem useful or relevant to some important outcome when in fact, it’s useless and irrelevant. In these cases, researchers find what they claim to have found, but what they find is not useful (despite them making it sound useful).

• Example: focusing on statistical significance when the effect size is so small that the result is useless. Clinicians often distinguish between “statistical significance” and “clinical significance” to highlight the pitfalls of ignoring effect sizes when considering the importance of a finding.

4. Hacking Beauty: make a result seem clean and beautiful when in fact, it’s messy or hard to interpret. In these cases, researchers focus on certain details or results and tell a story around those, but they could have focused on other details or results that would have made the story less pretty, less clear-cut, or harder to make sense of. This is related to Giner-Sorolla’s 2012 paper Science or art: How aesthetic standards grease the way through the publication bottleneck but undermine science. Hacking beauty sometimes reduces to selective reporting of some kind (i.e., selective reporting of measures, analyses, or studies) or at least of selective focus on certain findings and not others. This becomes more difficult with pre-registration; if you have to report the results of planned analyses, there’s less room to make them look pretty (you could just say they’re pretty, but that seems like overclaiming)

• Example: emphasizing the parts of the result that tell a clean story while not including (or burying somewhere in the paper) the parts that contradict that story

Science faces multiple challenges. Over the past decade, the replication crisis and subsequent open science movement have greatly increased awareness of p-hacking as a problem. Measures have begun to be put in place to reduce p-hacking. Importance Hacking is another substantial problem, but it has received far less attention.

If a pipe is leaking from two holes and its pressure is kept fixed, then repairing one hole will result in the other one leaking faster. Similarly, as best practices increasingly become commonplace as a means to reduce p-hacking, so long as the career pressures to publish in top journals don’t let up, the occurrence of Importance Hacking may increase.

It’s time to start the conversation about how Importance Hacking can be addressed.

If you’re interested in learning more about Importance Hacking, you can listen to psychology professor Alexa Tullett and me discussing it on the Clearer Thinking podcast (there, I refer to it as “Importance Laundering,” but I now think “Importance Hacking” is a better name) or me talking about it on the Two Psychologists Four Beers podcast. We also discuss my new project, Transparent Replications, which conducts rapid replications of recently published psychology papers in top journals in an effort to shift incentives and create more reliable, replicable research. If you enjoyed this article, you may be interested in checking our replication reports and learning more about the project.

Did you like this article? If so, you may like to explore the ClearerThinking Podcast, where I have fun, in-depth conversations with brilliant people about ideas that matter. Click here to see a full list of episodes.