Comments on: Can you trust survey responses? https://www.spencergreenberg.com/2025/03/can-you-trust-survey-responses/ Tue, 13 May 2025 07:41:21 +0000 hourly 1 https://wordpress.org/?v=6.9.4 By: Emiel https://www.spencergreenberg.com/2025/03/can-you-trust-survey-responses/#comment-53831 Tue, 13 May 2025 07:41:21 +0000 https://www.spencergreenberg.com/?p=4353#comment-53831 I would stil not trust self-reports all that much though. Whilst you point out great points here and in other similar articles about self-reports. I think that psychology as a science should lean a bit more heavily on including internal controls and be more concerned with standardizing and validating measures before stating what is and what is not trustworthy, as often the work has not been put in to test how reliable they actually are in specific situations. Considering how weak the correlations tend to be, before you can say anything useful about people in a whole I’d presume you’d need stable reliability across studies and across researchers. And from what I read, this is not the case. And then you have the problems of sampling that are often also not looked at deeply.

While I do think self-reports could be useful I think they need a lot more work before being considered a reliable tool.

]]> By: Emiel https://www.spencergreenberg.com/2025/03/can-you-trust-survey-responses/#comment-53830 Tue, 13 May 2025 07:36:02 +0000 https://www.spencergreenberg.com/?p=4353#comment-53830 I think one important thing to note is that correlations of 0.3 and 0.4 should not be considered quite strongly. I know psychology has a tendency to significantly lower the standards of effect sizes to what is considered useful, but even then the predictive value of r = 0.3 – 0.4 is low (it explains about 9 – 16 of the variance) and I think at best you could call it weak to moderate. If you mean that it is surprising, considering the problems of self-reports that such correlations are possible with point 3 and 4, I agree with that. Is that the point here? As I often struggle with vague or ambiguous language, so it could well be that I misunderstood the point.

]]>