Self-reporting on surveys seems ridiculously unreliable. People can lie or may not pay attention. People misremember things. People often lack self-insight. And YET, self-reporting fairly often works remarkably well in measuring things. Here are some examples:
(1) In a large study we ran, IQ (measured by performance on intelligence tasks) had a strong correlation with self-reported (remembered) performance on the math portion of the SAT exam (r=0.61, n=714), which most participants would have taken MANY YEARS prior. This suggests that they were neither lying that often nor were their memories (of a long-ago event) that bad. Though I think there was likely some inflation in their self-reported scores, their reported scores contained a lot of signals.
(2) When we asked people how much they agreed with the extremely subjective statement “I feel satisfied with my income” (on a 7-point scale), their responses correlated quite strongly with their self-reported household income (r=0.43, n=639). This suggests that self-reported subjective feelings can map pretty well onto actual realities.
(3) When we asked, “Have you ever been diagnosed with depression by a medical or mental health professional?” it had a reasonable correlation with the level of agreement to the single statement, “I often tell myself that ‘I am not good enough’ (r=0.37, n=509). This suggests that agreement with vague, seeming statements can say a surprisingly large amount about a person.
(4) You might think that even if it’s an anonymous survey, people wouldn’t be willing to admit socially undesirable behaviors or traits, socially stigmatized events, or highly personal things. But, in our experience, many people are willing to indicate that these apply to them. In one study, when asked if they had ever cheated on a romantic partner, we found that 32% of respondents admitted to having done so. In another, when asked whether, in childhood, an adult in their home ever hit, beat, kicked, or physically hurt them, 26% agreed. When asked if they had been raped, more than 20% agreed. While it’s certain that some people lie, many people are willing to honestly talk about their experiences in anonymous surveys.
While it’s important to be cautious about self-reporting on surveys – people can lie, they may not be paying attention, they may not remember, and they may lack self-insight – in my experience, it often works (perhaps surprisingly) well to tap important traits!
This piece was first written on March 29, 2025, and first appeared on my website on April 24, 2025.
I think one important thing to note is that correlations of 0.3 and 0.4 should not be considered quite strongly. I know psychology has a tendency to significantly lower the standards of effect sizes to what is considered useful, but even then the predictive value of r = 0.3 – 0.4 is low (it explains about 9 – 16 of the variance) and I think at best you could call it weak to moderate. If you mean that it is surprising, considering the problems of self-reports that such correlations are possible with point 3 and 4, I agree with that. Is that the point here? As I often struggle with vague or ambiguous language, so it could well be that I misunderstood the point.
I would stil not trust self-reports all that much though. Whilst you point out great points here and in other similar articles about self-reports. I think that psychology as a science should lean a bit more heavily on including internal controls and be more concerned with standardizing and validating measures before stating what is and what is not trustworthy, as often the work has not been put in to test how reliable they actually are in specific situations. Considering how weak the correlations tend to be, before you can say anything useful about people in a whole I’d presume you’d need stable reliability across studies and across researchers. And from what I read, this is not the case. And then you have the problems of sampling that are often also not looked at deeply.
While I do think self-reports could be useful I think they need a lot more work before being considered a reliable tool.