My hypothesis is different. I think that the effects of social media are extremely heterogeneous based on app, population, and dosage: that in some subgroups, some social media apps (when used in high doses) have substantially negative effects on mental health, but in other subgroups, using other social media apps in moderate doses has no negative effect on mental health.

For instance, 13-year-old girls in the US using TikTok or Instagram for 4 hours a day may be very differently impacted than 25-year-old men in Denmark using Twitter/X or WhatsApp for 30 minutes per day.

The current studies may be like trying to answer the generic question: “Do non-prescription drugs have a negative mental health effect?” This question can’t be answered because it combines too many dissimilar things. In particular, the answer hinges on which drugs we’re talking about (cannabis vs. fentanyl), the age of the person doing the drug (teenagers vs. adults), and the quantity of drug use (occasional vs. extreme usage).

If my hypothesis is true, then getting to the bottom of the true impacts of social media on mental health will require carefully designed studies that subdivide by app and by population (ideally after preliminary research is done to figure out what apps and which populations are reasonable to group together – for instance, it may be essential to segment by gender and rough age group and even by culture, but it’s important to get these segmentations right if the research is going to make progress).

Another thing that makes this research so tricky is that social media literally adapts itself to what you pay attention to. So if you tend to click on upsetting things, it will show you more upsetting things, which can create a self-reinforcing cycle, whereas if you click on things that are interesting and pleasant, you’ll get more of those instead. So even at the level of the individual, social media can provide highly varied experiences. It’s instructive to compare your social media feed to a friend’s (on the same app). When I’ve done this, it’s been remarkable to see just how different our experiences on that app are.

Overall, my best guess is that most people’s social media use would be found to have little or no negative causal link to mental health. But I would predict that there is a moderately sized causal negative link to mental health for:

• teenage girls scrolling Instagram a large amount (e.g., checking it >25 times daily)
• teenage boys playing video games (but not with friends), very large amounts (e.g., > 5 hours per day)
• people who are already predisposed to worry a lot about the state of the world, scrolling Twitter a large amount (e.g., > 3 hours daily)
• I also would predict a negative impact on attention or focus for those who use TikTok a lot (>5 hours daily)
• But I would predict little to no average negative mental health effects for apps that a person uses only 20 minutes per day or less, since I think that’s unlikely to be a high enough dose to cause problems for many people

Another hypothesis is that insofar as social media causes negative mental health impacts, it’s because it changes the situation for everyone at once. For instance, if all teenagers in a school are on social media, that can change the way that they socialize (or how much people socialize) and how they interact (e.g., how much bullying or social comparison occurs). By this view, studying what happens to individuals when they use more or less social media misses the important effects. If this is the case, it makes the phenomena even harder to study!

This piece was first written on August 29, 2025, and first appeared on my website on November 2, 2025.

Below is my attempt to summarize patterns across many anecdotal reports. I’d be curious to hear what you think, especially if you currently or have ever suffered from a painful or unpleasant chronic condition that modern medicine doesn’t have good solutions for.

While the evidence on this topic is extremely preliminary, what I say here could turn out to be wrong, and high-quality randomized controlled trials are desperately needed before we can be confident in these approaches being useful, anecdotally, there appears to be a pattern where people with these conditions are reporting substantial benefits (and sometimes even full recovery) from a combination of psychological and behavioral strategies – sometimes even people who have suffered for a decade or longer.

For those who have already tried all the obvious things, ruled out dangerous medical conditions, exhausted all the options presented by knowledgeable doctors, and don’t know what to do next, these ideas may be worth a try.

Important Note: none of what’s below implies that the person in question was never suffering from a disease, or that their pain is any less “real” than any other pain. Additionally, even if this approach works for some people, it will, of course, not work for everyone, and it may even make some people worse, so please explore with caution.

—

From what I can tell, the often-repeated common threads reported in anecdotal accounts of those who recover from these conditions (that they attribute their recovery to) appear to have three major elements:

—

Element 1: Foundation (this part is boring but important)

A focus on getting the healthy life basics in place, to set yourself up for potential recovery and to help you feel as good as you can (despite the pain). This often includes elements like:

• cutting out junk food and excessive sugar and replacing them with healthy, whole foods, and drinking sufficient water

• focusing on getting enough, high-quality sleep

• daily stress reduction, such as through a daily meditation practice, progressive muscle relaxation, deep breathing, a yoga routine, or massage

• reduce or cut out drugs and alcohol

• daily sunlight through some spent outdoors, ideally in the morning

• whatever daily movement or exercise feels manageable (even if just a short, slow walk outside)

• scheduling enjoyable activities regularly and aiming to find joy in ordinary pleasurable moments

• regular social connection with people you care about or find interesting

• If you are dealing with a mental health challenge (such as depression or anxiety, which are both common for people suffering from painful chronic conditions), seeking treatment from an expert (e.g., a well-trained therapist who specializes in the condition you’re grappling with)

• find ways to explore and process your difficult emotions, whether it be talking with a therapist, a daily journaling habit, just taking a few minutes daily to sit and let yourself fully feel your emotions, or speaking regularly to a trusted friend who is happy to listen

• if you tend to be hard on yourself or engage in a lot of negative self-talk, explore developing self-compassion (treating yourself at least as kindly as you’d treat a friend, and showing yourself compassion like you would to someone you care about)

• getting yourself out of psychologically unhealthy situations to the best of your ability (whether it be an unreasonably demanding work situation, a person in their life who treats you very badly or makes you feel bad all the time, or a people-pleasing mentality of never saying ‘no’ even though you are carrying a huge burden already)

These items in 1, above, are not designed to cure your chronic pain or even to reduce the pain; they aim to set you up for the maximum chance of feeling better, and so are important. Also note that 1 doesn’t involve taking a boatload of supplements or eating a highly unusual diet.

It’s 2 that is the more novel, potentially critical piece:

—

Element 2: Reframing and Reprocessing

Completely reframing your perception of the painful and unpleasant bodily sensations. The goal is to:

i) See these sensations as your body attempting to send you a helpful signal (e.g., some people like to start thanking their body for giving them this signal because they know it’s trying to protect them)

ii) Perceive these sensations as a false alarm. The idea is that the symptoms do not actually mean you are in any danger, nor do they indicate a life-threatening disease (since we’re assuming that has already been ruled out). The symptoms also don’t mean that you are destined to feel bad all day, or that the activity you’re doing when the symptoms emerge is going to cause any lasting harm to you.

The attitude to bring here is not one of bracing against the pain, and not one of trying to fix the feeling.

The concept is that these unpleasant bodily signals (which probably began as signals connected to an injury or disease) have somehow become detached from any injury or disease. Unfortunately, the signals persist – whether due to misfiring in your nerves, your brain misinterpreting benign signals as being dangerous, associative learning (X has preceded Y enough that now X causes Y), or some kind of accidental signal reinforcement (e.g., by responding to the signal as if it’s a sign of danger, the signal gets perpetuated).

Here’s my metaphor for this way of seeing things:

Imagine that your pain or unpleasant bodily sensations are like the barking of a very loyal guard dog. You brought this dog into your home to protect you back when you used to be in a very dangerous area, and the dog was very helpful at that time, barking at the very real danger that was frequently around you.

Now you live in a safe area, with nothing important for the dog to bark at. But the dog desperately wants to be helpful, and only knows one way to do so. Due to his extreme overeagerness and an unrelenting focus on potential danger, the dog ends up barking constantly. Whenever it barks, despite the now safe environment, you subconsciously still interpret this as a sign of genuine danger (since that’s always what it used to mean). This constant barking leaves you constantly anxious, on edge, or in a heightened state, and may have downstream consequences on your body (such as impacting your ability to sleep well or digest food), and generally makes you miserable. Due to the well-meaning dog’s misguided attempt to keep you safe, the barking is ruining your life.

The idea, therefore, is to retrain yourself to view the chronic pain and unpleasant bodily sensations as completely safe. You can facilitate this by noticing when you’re having negative thoughts about the pain and gently letting them go, and by practicing observing the sensations neutrally, without judgment. You can even practice accepting the painful feelings exactly as they are (and when your brain jumps in to label the feeling as “awful” or tells you “you can’t handle it” you can acknowledge those thoughts, gently let go of them, and return to observing the feeling non-judgmentally. As many people (including myself) have experienced exploring this way of viewing pain during meditation, shifting our attitude toward pain can immediately reduce the suffering the pain causes us.

Redirecting to something positive after doing so is also something that people report as a useful addition (whether that’s some other part of your body that feels good right now, a humorous re-interpretation of the event, a pleasing visualization, or an activity that’s pleasant that you’d like to do now instead of focusing on the pain, etc.)

Importantly, for many people, their bodily signals really ARE indicating imminent bodily danger (e.g., if you have a broken bone in your foot, you may actually need to stay off it for a while to let it heal) – that’s typically how pain works. So if you’re considering trying these techniques, it’s important to first rule out that you’re in that group.

Additionally, it’s important to distinguish a bodily signal indicating true danger vs. one merely indicating “you need to rest”, which is not inherently dangerous, but is important to heed and not ignore. It’s not that you should learn to ignore bodily signals – many such signals provide us valuable information, and ignoring our bodies is a recipe for potential problems down the road. It’s instead about changing the relationship to our chronic painful bodily signals, such as experiencing them non-judgmentally with peace and acceptance, viewing them as our bodies attempting to give us useful information, and not responding to them as though they are dangerous.

For those interested in trying this approach, here’s the final piece of the puzzle:

—

Element 3: Practicing and Expanding

• While adopting the points from 2, above, gently and at a manageable pace, challenge yourself with whatever triggers your immediate (acute) symptoms. That is, test the waters with things that would normally make your symptoms immediately feel worse. Do so while maintaining the perspective that the symptoms are safe, they are there in an attempt to help you, and that they do not indicate any actual harm to you. Aim to view the symptoms neutrally and objectively without judgment and keep trying to accept them as they are, without needing them to be different. Then, once that becomes a bit easier, gently push the limits further toward somewhat more intense potential triggers and toward things you’ve been avoiding out of the fear of their impact on your symptoms. An important note: the idea here is NOT to push yourself more and more in an attempt to build up tolerance, strength, fitness, or resistance to fatigue (as one might try to do in Graded Exercise Therapy) – the idea, instead, is to practice reframing and reprocessing symptoms in progressively more challenging situations, while being careful to avoid overdoing it (which can lead to crashes). Many report that pushing yourself too hard, too fast, can backfire. So go slow, be gentle with yourself, and treat yourself with self-compassion.

Expect some setbacks along the way – progress is not likely to be linear, with lots of random daily variation. Be consistent, working at this daily but at a pace that feels comfortable, and track symptoms at the end of each day (e.g., pain level, fatigue level, stress level), which you can check over time to see whether there is a longer-term trend towards progress.

Part of the long-term goal with this is to end up paying less and less attention to your symptoms – since (in this case) they are not indicative of any actual problem, eventually you can learn to ignore them.

Note: gently pushing your boundary on symptoms doesn’t mean blowing past your limits. For instance, if you know that a 15-minute walk will leave you feeling fine at the time but cause you to crash for the next 2 days, then walking for 15 minutes would not be an example of what I’m describing.

—

What’s the point of this whole process? It’s really two-fold:

A) It can simply make it substantially easier to deal with the chronic condition, and mean that you suffer less despite all the pain, and live a happier life even as you have the pain. It may cause the pain itself to feel less bad (through a reframing of the pain), cause you to have less intense negative emotions about the pain (which means reducing the second-order pain and other consequences caused by these negative emotions), and expand the range of activities you engage in that make your life worth living.

This could be a good enough reason to try this approach. And that’s part of why I like this approach. While there’s always a risk of it backfiring, even if it doesn’t cause any huge change, I think it has a pretty good chance of making life more manageable and leading you to suffering less from the pain (i.e., even if the pain is not itself lessened, changing your perception of the pain can mean you suffer less from it).

But, here’s where things get weird and much more speculative:

B ) Some people find that after weeks or a few months of following processes similar to what’s described here, they are completely or nearly completely cured. These reports are still very much anecdotal, and much more rigorous research is needed to be confident in the cause of improvement for such folks (so this should all be taken with a big grain of salt), but it’s quite striking how many people who were sick for years or longer have reported rapid progress with approaches similar to what I’ve described here. Though the exact approaches they use differ, and have a variety of names, and a variety of distinct elements, I’ve included here aspects common to many of them.

Many people will only get benefit A) from this approach, and that alone may make it worth it. So it may be best to think of B) more like a speculative bonus that (based on many anecdotal reports) seems to happen for some people.

—

Obviously, there are other major caveats here: people could be wrong about this being the cause of their improvements, and even if this approach does cure some people, it’s unclear what percentage of people would be cured if they were to fully take on such an approach. Additionally, there are lots of people this is not appropriate for, such as people who have a life-threatening medical condition, people who have not yet had a thorough medical workup, people experiencing rapid weight loss or fevers, and so on. The evidence here is not strong, and could turn out to be wrong. Just because modern medicine doesn’t have a good understanding of a disorder doesn’t mean that it’s connected to psychology and behavior. And some people even report having tried this approach and had it backfire, where it left them worse off, so please be careful.

If this method does work, though, why does it work? It seems like one or more of the following explanations may be at play (which may vary depending on the individual):

• Pain signals and negative bodily sensations can occur in the absence of injury or disease (for instance, due to misfiring in the nerve, the brain misinterpreting signals, or other causes). A proof of concept where we know this happens is Phantom Pain Syndrome, where a person who has had a limb amputated may experience intense chronic pain that feels to them to be coming from that (now absent) limb, which, of course, it can’t actually be coming from.

• It may be that pain signals or unpleasant bodily sensations can become reinforced by our reactions to them (though this is not well understood). If so, the way we react may inadvertently cause pain signals to persist even past the point of the injury or disease that originally caused them to start. Perhaps somehow our attention to the signals, or our viewing them as dangerous, convinces our brain that the signals are worth sending.

• Or, a subtly but importantly different mechanism may be at play: our negative reactions to pain signals or unpleasant bodily sensations may leave us in a highly elevated state (e.g., anxiety or cortisol), and this elevated state may generate new pain signals or exacerbate existing signals (e.g., increase tightness in the stomach or an inflamed feeling in joints) or new unpleasant bodily sensations (e.g., trouble with digestion, difficulty breathing) which then generate more negative reactions, in a self-perpetuating cycle

• Or, a related possibility, is that your brain has somehow learned associations that are now triggering reactions. Perhaps your brain learned that when X happens, then the proper response is Y, and your brain is now triggering Y in response to X, despite that reaction being useless now and generating negative side effects for you.

• In some cases, behaviors we engage in to avoid pain may actually increase or perpetuate it – e.g., avoiding using a joint because it hurts may actually cause it to be stiff and painful from lack of use, which may cause us to continue to avoid its use.

Pain can be completely real, disabling, and have a biological origin, even in the absence of damaged or diseased tissue.

—

One important thing you may wonder: Is there actually evidence that psychological and behavioral approaches can improve the lives of people with these conditions? Well, while this field is, overall, woefully under-researched, there’s quite a bit of evidence that psychological and behavioral strategies can help (even though the nature of these strategies differs, and may only be partially overlapping with what I’m describing in this post). Here is a quick recap of some of that evidence:

(1) IBS: A meta-analysis of randomized controlled trials of psychological treatments for Irritable Bowel Syndrome looked at 15 studies. It found greater improvement in the psychological treatment groups compared to the control group on symptom severity, quality of life, and abdominal pain, but no difference in diarrhea or constipation. Another meta-analysis on psychological therapies for IBS looked at 41 randomized controlled trials and found that CBT and gut-directed hypnotherapy outperformed education and routine care.

(2) Functional Dyspepsia: A meta-analysis of psychotherapy treatments for people with Functional Dyspepsia. It looked at 5 studies and found that those receiving psychotherapy improved in gastrointestinal symptoms.

(3) ME/CFS: A 2011 meta-analysis of randomized controlled trials of treatments for Chronic Fatigue Syndrome looked at 16 studies on Cognitive Behavioral Therapy (CBT) and concluded overall that those receiving CBT had greater symptom improvement than control groups. As a commenter pointed out, those studies used an earlier definition of ME/CFS that didn’t require post-exertional fatigue. A later 2020 Systematic review of randomized controlled trials looked at 12 studies on CBT, and found that 4 showed it to be effective, 2 ‘Partially significant’, and 6 found no effect.

(4) Chronic pain: a Cochrane meta-analysis looked at 75 studies on psychological treatments for chronic pain conditions, including fibromyalgia and chronic low back pain. A number of types of psychological treatment were included, such as CBT, behavioral therapy, and ACT. They conclude: “On average, compared to people who receive no treatment for their pain, people treated with CBT probably experience slightly less pain and distress by the end of the treatment and six to 12 months later (moderate-quality evidence). They may also experience slightly less disability on average (low-quality evidence).”

So the evidence overall quite strongly indicates that psychological and behavioral strategies can help people with these conditions (though, unfortunately, there is little research that has been done directly testing the specific strategies I’m describing here, and there is a lot of heterogeneity in results, especially with ME/CFS).

The most direct evidence of a technique very similar to what I’m describing (that I’m aware of) comes from just one randomized controlled trial on a method known as “Pain Reprocessing Therapy” (PRT) for chronic back pain. It showed promising results (though more studies are very much needed). PRT has many common elements with what I’ve described in this post.

Of course, if you’re suffering from one of these chronic conditions, you may simply want to try Cognitive Behavioral Therapy if it seems promising to you or (if it’s a gut disorder) gut-directed hypnotherapy as there’s reasonable evidence (as described above) that they improve many people’s quality of life who suffer from chronic conditions, even if they don’t provide a cure for most people.

—

The best source I know of to learn about these many anecdotal accounts of people improving from these conditions is the YouTube channel of Raelan Agle. She had ME/CFS for 10 years and eventually recovered, and on her channel, she interviews people who have recovered from ME/CFS and Long Covid. These three videos in particular were very influential for what I wrote in this post:

• What Raelan says she learned from conducting 75 interviews with people who recovered from Long COVID or ME/Chronic Fatigue Syndrome.

• Raelan’s recovery themes from 200 Interviews:

• What Raelan says about why recovery experts disagree and what they agree on.

There are also interesting books related to this topic, such as “The Way Out” by Alan Gordon and Alon Ziv (which teaches a specific method known as “Somatic Tracking” that’s very related to, but not identical to, what I discussed in this post).

This piece was first written on October 30, 2025 and first appeared on my website on November 10, 2025.

You can create your first “data loop” by choosing a tool from your tool belt (say, conducting a series of interviews with your most loyal customers) to answer your Product Question. After absorbing much of what this tool has to teach you about this question, you’ll be hearing the same answers again and again. Data loops are ephemeral, and once one diminishes in value, it has served its purpose.

The next step is to ask yourself again: “What don’t I know that I must know?” The answer yields your next Product Question, which leads to your next tool to create another data loop (say, reading many articles by experts). Once that stops being as useful, you can continue constructing other data loops using other tools (e.g., conducting a survey, watching people use your product, getting criticism from User Interface experts, studying the statistics of user behavior, attempting to sell your product at different prices). For each Product Question, you can consider the many tools in the tool belt (e.g., user interviews, expert feedback, surveys, studies, A/B tests, analysis of user behavior, etc.) and pick the one that is most suited for answering it. If you practice this, you’ll become a master of the “Question Looping Process.” 

The Question Looping Process:

Step 1: Ask the Meta Question: “What’s the most important thing that we need to know that we don’t know?” 

Step 2: Answer the Meta Question. The answer to it is the Product Question (e.g., “Why are users dropping out during our onboard process?” or “What are the triggers or motivators that get our power users coming back?” etc.)

Step 3: Ask the Tool Question: “Which tools in the toolbelt will most effectively and efficiently help us answer the Product Question?” For instance, it could be a survey of existing users, interviews with existing users, reading articles by practitioners, studying competitor products, analyzing the behavior of existing users, etc.

Step 4: Use the selected tools to answer the Product Question.

Step 5: If the Product Question is now answered, make product improvements and collect data to make sure that the question’s answer was correct (for instance, by doing A/B tests of the new product improvements or by doing further interviews after the product improvements were made). If the Product Question is not answered to a sufficient degree, return to Step 2 and re-ask the Tool Question. If you’re no longer confident that the Product Question is really the right question to be asking, return to Step 1 and re-ask the Meta Question. Or, if you’ve succeeded at making product improvements, you should also return to Step 1.

The Tools In the Tool Belt

Okay, but what are the tools in the tool belt? If you want to improve the quality of a product, it’s extremely valuable to be aware of all the tools at your disposal and to choose the right tool to answer each question you have. If you focus too much on just one or two tools, you’ll miss out on important insights. For each question that you are trying to answer about the product or content, you should think about which tool in the tool belt is best for the job, just like a mechanic must think about whether a hammer or wrench is more appropriate for what they are trying to get done.

Important note: a really common issue is that people start with the tools that they are familiar with or have easily available rather than with the questions they want to answer. They’ll think, for example, “We have customer data available, so let me analyze that to figure out what to do with our product.” The problem is that this often either leads to (i) focusing on whatever question you happen to be able to answer easily with the tool you chose rather than the question you should be trying to answer or (ii) trying to use the wrong tool to answer the right question. It’s much better to start with the important questions you have (e.g., using the question looping process above) and THEN consider what tools you have at your disposal to answer those questions, choosing the best tool or tools to answer the most important questions you actually have. If you have, for example, a bunch of data, there’s nothing wrong with spending some time pouring over it; maybe you’ll see something interesting. However, this undirected exploration is not a substitute for the Question Looping process.

With all of that said, these are the tools in the content developer’s tool belt. Learn to master as many as you can in order to have the greatest chance of making an amazing product!

1. Intuition/Experience

i. Leverage your own intuition and experience when looking at the product (e.g., if the problem is that people are dropping out of the onboarding process, maybe the reason will jump out at you when you look closely at the onboard).

ii. Adopt the user’s perspective! Really try to simulate the mind of the user while seeing the product or content with fresh eyes (take it in as though you’ve never seen it before). Ask, on each page, “What would a real user be thinking? What would they be confused by? What would they notice first?” Your success at this depends on your ability to (a) see the content with fresh eyes, as though you’ve never seen it before, and (b) really model and simulate the mind of a user.

iii. Seek team feedback. It’s usually best if one to three people give feedback at a time, using an agreed-upon format.

a. Consensus may be enough to move forward.

b. If you do not reach a consensus, you may want to consider:

1) Who has the most experience (with feedback) on this topic?

2) Who has the best design eye?

3) Who can best simulate the user’s mind?

4) Who has a track record of being right about this sort of question?

iv. Seek beta-tester feedback. Beta testers are people that are real users (or similar to real users) who are also motivated to help you improve; they tend to give more detailed and thorough feedback than most real users would. It can be really powerful to build a beta tester list and seek their feedback.

2. Interviews

i. Select people from the target user group who have never seen the product, then interview them about their experiences (e.g., ask them how they currently solve the problems that the product is designed to solve, ask them what would make them hesitant to try the product, etc.).

ii. Watch people in the target user group who have never seen the product use the product for the first time (e.g., watch them interact with the product and have them vocalize all of their thoughts and reactions while they go through the onboard process and try the product for the first time).

iii. Interview new users who have used the product for a short amount of time but who are not yet power users.

iv. Interview former users who used the product for a significant amount of time before ceasing to use it (e.g., ask them about why they stopped using it, what the product lacked, in what ways their goals or needs weren’t met, which features might make them want to use it again, etc.).

v. Interview power users who are getting the most value from the product and really love it (e.g., to discover what brings them back to the product and how it could be made even more valuable for them).

vi. Interview stakeholders (e.g., the people that buy the product for others, such as HR departments or others involved in distributing the product but who don’t use it directly).

3. Surveys (qualitative surveys, quantitative surveys, and mixed qualitative + quantitative surveys)

i. Survey people who have never used the product. 

ii. Survey real users within the onboarding process.

iii. Survey real users while they are in the middle of using the product.

iv. Survey real users who have used the product for a while.

v. Survey former users (who stopped using the product).

vi. Survey stakeholders (e.g., those who purchase on behalf of others).

4. Objective data and usage statistics

i. Analyze usage data from new users (e.g., how do new users interact with the product?).

ii. Analyze usage data from former users (e.g., how do people who stop using the product behave?).

iii. Analyze usage data from power users (e.g., how is the product used by new users who love the product and use it the most?).

iv. Use “alarm system” metrics (i.e., metrics that are designed to only change when something is very wrong with the product).

v. Use “success metrics” (i.e., the key metrics about the product that you’re aiming to improve, such as growth or retention metrics).

5. Reading

i. Learn what academic papers say about how to achieve X.

ii. Learn what articles by experienced practitioners or experts say about this topic.

6. Studies

i. Run a paid usage study (pay people to use the product for a certain amount of time; collect feedback throughout the process and/or include a feedback survey or interview at the end).

ii. Run a paid study on one small part of the product (e.g., have users interact with just a part of the product or a new user interface, then collect feedback).

iii. Run an A/B test to see which versions of a feature or page lead to better outcome metrics.

iv. Run randomized controlled trials (randomize some people to use your product (or to use one version of your product) and randomize others to a control group, then collect follow-up surveys or metrics after some interval).

7. Expert feedback (academic, practitioner, or paid consultant)

i. Collect feedback from experts looking at the product directly (or observe their reactions to the product)

ii. Ask experts to perform a pre-mortem (e.g., “Suppose this doesn’t work. What’s your prediction as to why?”).

iii. Ask experts “Would you do…?” questions (e.g., “If your goal was to take what we have now and make it better at achieving X, what would you do?”).

8. Behavior change frameworks

i. Use the Ten Conditions for Change (or one of the 17 frameworks summarized at the bottom of the Ten Conditions for Change website).

9. Market research

i. How does successful product Y achieve X? What are the successful products (e.g., products that are growing more quickly than others) doing well that we should consider emulating? What are the things that seem important to users that existing products are doing a poor job of addressing, and could this present an opportunity for us to address the problem?

ii. What do existing trends, changes, and market forces tell us (e.g., what’s the adoption curve on new technologies)?

iii. What do societal or publicly-available statistics tell us about people’s purchasing behaviors, other behaviors, and beliefs? 

10. Quality Assurance Testing

i. Ask team members to simulate being a real user.

ii. Standardize Quality Assurance (QA) test procedures (create checklists).

iii. Run bug bounty programs.

iv. Give current users an easy way to report issues.

v. Automate checks (e.g., Grammarly for spelling and grammar).

vi. Run copy editing passes.

At least some of these steps should be built into a standardized process when developing content. For instance, you may want to have every piece of content checked over by an expert at some point in the process, and you may want to have a standard quality assurance testing process.

Here’s an example standardized process that might be implemented for producing high-quality content:

This piece was first written on November 23, 2019, and first appeared on this site on August 26, 2022.

I’ve included the details of the event format below, including materials you can use to throw your own version of it.

Important Note: this event format requires at least ten attendees to show up (I recommend at least five males and five females) so as to avoid de-anonymizing participants. When I ran it there was about 20 people, which is close to ideal, I think.

GOALS

The goals of this event are:

• to help attendees understand each other on a deeper level (including potentially gaining some insight into human nature in general)

• to enable attendees to see how their thoughts, feelings, and behaviors are similar or different to those around them on topics that typically are taboo to discuss

• to have an interesting and enjoyable time.

I got a very positive response to this event format from those who attended.

THE FORMAT IN BRIEF

• Attendees will be asked to completely honestly fill out an anonymous survey at a minimum of 24 hours before coming to the event, which will contain many controversial, taboo, embarrassing, uncomfortable, and rarely asked questions.
• You’ll then compile the responses (grouping them by question in a totally anonymous way and calculating the % of people that gave each response for multiple-choice questions).

• You’ll hand out printed versions of these responses to attendees when they arrive. After the attendees have time to read through all the responses, you’ll then go question by question, having a quick group discussion about the responses to each.

• At the end, you’ll discuss as a group what you all learned from the experience.

• Finally, you’ll collect the sheets from everyone and shred them so that no one has a record of people’s private responses.

HOW TO DO IT

Step 1: Make a copy of my online survey containing taboo, controversial, and rarely asked questions (using the survey of questions I already created is the easiest option – preview all the questions here: http://bit.ly/2o5GBjq, and make a copy of the survey here: http://bit.ly/2Cm9oVH so that you can send it out). Or, if you have more time and want to customize the questions, you can crowdsource them from attendees in advance and put them on a survey platform that doesn’t record any personally identifying information (i.e., no email address, no IP address). If you plan to crowdsource questions, you should send out an anonymous survey to attendees beforehand to ask what questions they would want to be answered by other attendees (anonymously) that they would never normally be able to ask (here is my survey for collecting these questions from attendees: http://bit.ly/2BYQDvM and you can make your own copy of this survey here: http://bit.ly/2EqVszY). Once you get the questions back, you’ll turn them into another anonymous survey (where you actually ask those questions) which you will also send out to attendees before the event (being sure to give yourself plenty of time to process the responses to prepare event materials).

Step 2: Send an invite to potential attendees (sample invitation down below), including a detailed explanation of the event format (so they can knowingly opt-in rather than having it sprung on them). If you have the survey questions already, you should include them in the invite as well. Make sure to make it clear what the deadline is for fill out the survey (I recommend setting it to be at least 24 hours before the event starts so that you have time to prepare the results). Be sure to send a reminder before this deadline to increase the percent those who respond.

HOW I CONSTRUCTED THE QUESTIONS

I constructed my set of “taboo, controversial and rarely asked” questions (available at my survey link above) using the following process:

A. I crowdsourced question ideas from all the people who planned to attend the event.

B. I brainstormed additional ideas on my own that seemed like they could be interesting.

C. I then ran a study where 100 people in the U.S. on the Amazon Mechanical Turk recruitment platform were asked to answer each question anonymously. I ended up with way too many questions for the final survey.

D. I winnowed them down by reading all the responses of people on Mechanical Turk, retaining for the final survey those questions that seemed to me to produce especially interesting or surprising or discussion-worthy results (see link below for anonymous Mechanical Turk user responses if you’re interested).

PREPARING FOR THE EVENT

After you’ve received everyone’s survey responses, you’ll want to prepare handouts for each participant with all of those responses in completely anonymized form. It is critical that you never read any one individuals responses while you’re preparing the materials. Group the responses immediately by the question, and shuffle their order, so that you can only ever see the responses for all respondents as a group. This helps ensure the anonymity of every respondent. For multiple-choice questions, you should calculate the percent of people that gave each response (I put these on PowerPoint slides which I presented to the group, but you could put them right on the handouts). For free form text responses, you’ll want to have a list of all responses to each question, grouped by question. Be sure to shuffle the order so that the 1st response to each question is not the same person each time (otherwise, participants may accidentally be de-anonymized).

If you use the exact questions that I did, you can also show alongside your group’s responses, the responses from Mechanical Turk respondents, which you can find here in the anonymized form: http://bit.ly/2HgzX2s. I recommend only doing this for the multiple-choice questions, as otherwise, the handouts might become unreasonably long (expect to give at least seven pages of handouts if you have 20 attendees and you use a small font and are smart about space on the page).

OPTIONAL: USING MY PYTHON CODE TO PROCESS RESPONSES

If you use my exact survey, you can also use python code (download it here: http://bit.ly/2Hgi6su) that I wrote to make the process of preparing responses easier and prettier (you’ll need to use Python 2.7 to use it, which you can easily install on Mac or PC). You’ll also need to install the pylab and text wrap libraries (first install pip, which is a python installation system, then you can easily install these using pip). The way the code works is that it takes two input files: one has the responses from mechanical Turk workers (which is already provided for you in the folder you download, so there is nothing for you to do), and the other is the CSV file you download yourself of your attendees’ responses to the survey (you can get to this download option from the edit page for your copy of my survey, just click on data and you’ll see a download button). Place that CSV file you downloaded into the “python code” folder once you unzip the code (usually, you can unzip just by double-clicking). Then change the line of code comparisonFileToProcess=”…” within the file prettyPlotYesNo.py so that the … is replaced with the name of your CSV file. Finally, run the python file prettyPlotYesNo.py by navigating to that folder in terminal (Mac) or Cygwin (Windows) and doing:

python prettyPlotYesNo.py

It should then automatically fill the folder “figures” with all the results (including both figures and text files of qualitative responses). The skinny bars will represent the responses of mechanical Turk workers, and the thick bars will represent the responses of your attendees, so you can compare them side by side (i.e., you can see how representative your attendees’ responses were).

AT THE EVENT

Hand out printed sheets that show the responses of all the attendees to each of the questions. Give people 10 minutes or so to read them all (to form initial impressions and reactions). You may want to provide pens so that people can jot down notes as they go or mark responses that were especially interesting. When people are done, start with the first question, have everyone quickly read the responses to that single question again to remind themselves what others said (if it was a qualitative question) or have them look at the percent of people that gave each response (if it was a quantitative question), and then have a short group discussion about the responses to that question. If people are quiet, you may want to ask the room questions like:

•What was your reaction to reading people’s responses to this question?

•What surprised you about the way people answered the question?

•What patterns did you notice in people’s responses?

After a few minutes, or if people no longer seem to be engaged in the discussion, or if the discussion has gone on a tangent, move onto the next question (having everyone quickly read the responses to that next question again before discussion ensues). Repeat this until you’ve gotten through all the questions.

At the end, ask people what they feel like they learned overall from the experience of seeing attendees answers to these questions and discussing them as a group. When we ran this, people had a very positive reaction to the experience!

Be sure to collect all the handout materials back at the end and shred them. Make sure everyone knows at the beginning that they will not get to keep the sheets of people’s responses and that they should hand them back to you before leaving to protect the private responses of all attendees.

SAMPLE INVITATION TO SEND FOR THE EVENT

I’d like to invite you to an event called “Anonymous Answers to Anonymous Questions,” where you’ll get to find out the (anonymous) answers that all the other attendees have to controversial, taboo, embarrassing, uncomfortable, and rarely asked questions.

Here’s how the format works in detail. Please read these instructions carefully if you plan to attend, and let me know that you’ll be coming!

Before coming, all attendees will fill out an anonymous survey (see link below) with a bunch of controversial, taboo, or otherwise unusual questions. At the beginning of the survey, you and the other attendees will be asked to swear that you will respond completely honestly.

When you arrive at the event, you’ll get to see all the responses given by other attendees to all the questions (in a completely anonymized format). We’ll then go question by question, reviewing them together, and discuss them as a group (without knowing who gave which answer)! You will not be able to take any materials with you when they leave because they will contain the personal responses of group members.

I explored this question by running two little studies across two different populations: mechanical turk (“mturk,” n=49, all U.S.) vs. my Facebook friends (“FB,” n=111, who tend to be in the U.S.) with different approaches, me hand categorizing written responses vs. participants selecting the category that best matches what they wrote using 24 categories in random order developed from the first study, including an “other” option.

Across the two studies, 71% of free form written responses to the question of “what you want more than anything else in the entire world” fell into just these nine categories:

1. a romantic partner (17% FB vs. 6% mturk)

2. the wellbeing of my family (5% FB vs. 17% mturk)

3. happiness (7% FB vs. 15% mturk)

4. money (8% FB vs. 14% mturk)

5. a markedly positive effect on the world (18% FB vs. 0% mturk)

6. good health (8% FB vs. 6% mturk)

7. a family or children (5% FB vs. 2% mturk)

8. a job with meaning (5% FB vs. 2% mturk)

9. freedom (5% FB vs. 2% mturk)

In the Facebook version, I also asked people to come up with something they could do in the next few days to increase the chance that they eventually get this thing they most want. Surprisingly, 95% of people were able to come up with something that they thought would increase their odds! When asked when they would do this thing, they came up with that would increase their odds of getting what they most want, 75% of people said they would do this thing within the next week, and 62% said they would do it within two days!

Tentative lessons from this data:

(1) The things that people most want in the U.S. tend to fall into just a small number of basic categories: a romantic partner, wellbeing of family, happiness, money, positive impact, health, family/children, meaningful work, freedom. In the mturk study, these nine categories covered 64% of responses, and in the FB study, 78% of responses.

(2) The median time for the entire FB study was 3 minutes, so if you take 3 minutes to think, you can decide how to take action toward the things you most want.

(3) There’s a good chance you can start really soon on this strategy to increase your odds of getting what you most want (e.g., within the next couple of days or at least within a week).

(4) If you have the nine things in the list below (or almost all of them), consider yourself very lucky! You have what people want! Extract every sliver of joy and meaning from what you have while you can!

If you’d like to try this procedure yourself in an easy, step by step fashion, here’s the link to the FB version of the study: http://bit.ly/2DtQQoK

Here are 24 different methods you could use to better understand the way humans work. In each case, I use “trying to figure out new things about post-traumatic stress disorder (PTSD)” as an example. What techniques am I leaving out here?

1. Papers: Reading academic studies (e.g., from social psychology, consumer psychology, cognitive psychology, neuroscience, etc.)

Example: reading the newest academic papers on treatments for PTSD.

2. Products: Carefully examining products and apps that have been successful (as well as, potentially, those that have failed) to understand what human needs they were or weren’t satisfying and why they succeeded or failed.

Example: looking at what mental health apps related to PTSD are popular in the app stores.

3. Outliers: Investigating extreme case studies of real-life events (e.g., case studies of people with extreme psychological conditions who may have been reported about in the news or true stories about what actually happened when people were placed into really extreme conditions).

Example: reading about cases of extreme cults and whether or not each cult produced PTSD-like symptoms in its members.

4. Society: Examining the large-scale trends and organization of societies, including potentially human cultures around the world (American culture, traditional cultures, etc.) and how they are similar or different, and which societal structures tend to be created and what the effects of these structures are, etc.

Example: investigating which sort of societies PTSD appears to be much more prevalent in.

5. Conversations: Talking to smart and insightful people about what they think is true of human psychology. You can also show them the models or theories you have so far to get their criticism and suggestions for improving them.

Example: asking really smart (non-experts) who had PTSD what their current theories about PTSD are.

6. Reading: Reading books, blog posts, media articles, etc., containing information from smart or insightful people about psychology (including potentially writing from life coaches, relationship experts, marketers, thoughtful bloggers, user experience designers, game makers, etc.)

Example: seeing what theories bloggers have proposed regarding PTSD that many academics may not have considered.

7. Experts: Reaching out to experts and talking to them to understand their models of psychology (e.g., psychologists, psychiatrists, game developers, etc.). You can also show them the models and theories you have so far to get their criticism and suggestions for improving them.

Example: talking to world-class experts in PTSD.

8. Textbooks: Reading academic textbooks about psychology, cognitive science, etc.

Example: Linehan’s textbook on DBT and thinking about possible applications of DBT in PTSD treatment.

9. Models: Looking at what models or frameworks of psychology others have created (e.g., in social psychology, theoretical cognitive neuroscience, gamification research, persuasion research, etc).

Example: looking at the model of PTSD that is used in Cognitive Behavioral Therapy treatment.

10. Intuition: Our own intuition for what’s true about humans (presumably based on our first-hand experience interacting with others, as well as our first-hand, ingrained experience of just being a human).

Example: We may have the intuition that after experiencing a traumatic event, people will tend to start having frequent disruptive thoughts about that event, even when doing unrelated things.

11. Interviews: talking to individuals or groups about their thought processes, feelings, behaviors, etc., to understand their individual psychology and then attempting to extrapolate that knowledge to humans more generally.

Example: interviewing 20 people with PTSD about what their experience of it is like and what does or does not seem to help them.

12. Application: attempting to put into practice a psychological theory (e.g., by coaching individuals using a psychological technique or by trying to build an app that implements it) and paying attention to the ways it seems to work or not work.

Example: creating a tool for people with PTSD and making it publicly available, then monitoring how people respond to it.

13. Anecdotes: psychologically surprising or interesting anecdotes that we’re confident actually happened (e.g., because they happened to us or were reported on by a trustworthy source) that may have a bearing on human psychology.

Example: a story a friend told you about how they (believe) they fully cured their own extreme PTSD in a matter of days.

14. Self-observation: careful (and honest) observation of our own thoughts/beliefs/behavior/emotions and internal processes.

Example: observing the mental processes in your own mind that seem to occur after a very upsetting event has happened.

15. Self-experimentation: simply trying techniques and paying close attention to what effects they appear to have and what it feels like to apply them.

Example: try the technique of writing down for a week every upsetting thought that you notice yourself having, as well as writing down everything you observe or learn about the experience of doing this.

16. Deduction: combining propositions we already believe to produce new propositions we didn’t realize before that may be true.

Example: knowing that women with PTSD are much more likely to have associated depression than men with PTSD, and knowing that depression is a strong risk factor for attempting suicide, we might predict that women with PTSD are more likely to attempt suicide than men with PTSD (even though they may not be more likely than men to actually commit suicide).

17. Experiments: Conduct your own online randomized controlled trials, surveys, longitudinal studies, or fMRI studies, etc.

Example: looking at which variables that don’t seem obviously PTSD related are, in fact, strong predictors of whether someone has PTSD.

18. Statistics: looking up known statistics about a phenomenon.

Example: looking up whether younger people or older people are known to be more likely to get PTSD in any given 12-month period.

19. Data: finding existing data sets (e.g., large longitudinal surveys or government data sets) and running your own statistical analyses to test hypotheses.

Example: measuring whether it is true that after someone gets a pet, they are more likely to recover from PTSD than similar people who do not get a pet. Running studies can also be a good way to search for new hypotheses. For instance, examining which of many variables in the data are most associated with a rapid recovery from PTSD.

20. Literature: looking at portrayals of psychology in literature, art, myths, stories, philosophical writings, etc.

Example: What hypotheses do Kafka’s novels give us about the nature of trauma?

21. Crowdsourcing: Request suggestions for hypotheses on a social media platform like Facebook or X or on a question-answer site like Quora.

Example: posting to Facebook asking people to suggest theories for why PTSD sometimes does and sometimes doesn’t happen after extreme trauma.

22. Social experiments: bringing a group of people together under a new set of chosen social rules or guidelines and observing what happens.

Example: getting a small group of people together with the idea that it will be a time to discuss traumas that members of the group have experienced.

23. Evolution: considering what possible evolutionary function different psychological phenomena might have.

Example: consider ways that aspects of PTSD responses might be evolutionarily adaptive.

24. Practitioners: examine how people “in the field” who benefit from being able to understand this aspect of psychology do their work.

For example, at veterans hospitals where rates of PTSD are high, investigate what techniques and approaches the therapists have developed to work successfully with their patients.

This piece was first written on November 16, 2017, and first appeared on my website on July 22, 2025.

The results of the predictive model surprised me! 

Before scrolling down, take a moment now to try to guess the top 5 variables that you think would be useful in predicting the severity of a person’s depression! I tried to include a very broad range of demographic and personality variables in the model – testing 91 in total.

Important note: these variables are predictive, not necessarily causal. In each case, we aren’t certain whether a higher incidence of that variable leads to worse depression, whether more severe depression causes a higher incidence of that variable, or if some outside factor happens to cause both depression and that variable. Also note that the variables shown below those that were most predictive of depression when controlling for all the other 90 variables. That means that a one standard deviation increase in each of these variables was associated with a greater increase in PHQ9 scores than any of the other variables (when training a model with all the variables at once). But all but of these variables are all associated with depression in the same direction on there own (when not controlling for other variables) except for one exception that is mentioned.

Variables That Most Strongly Predicted Depression

(1) Introversion (related to the Big 5 personality trait of Extroversion), as measured by the statements: “I see myself as extraverted, enthusiastic” or “I see myself as reserved, quiet”

(2) Feeling Poorly Rested after sleeping their ideal number of hours, as measured by the question: “Typically, how rested do you feel upon waking when you’ve just slept your ideal number of hours?”

(3) Under-sleeping, that is, the amount of time lower than their ideal hours of sleep that they sleep per night, calculated by subtracting actual average hours of sleep from reported ideal hours.

(4) Poor Treatment From Caregivers in childhood, as measured by the question: “Overall, how well were you treated by the people who raised you (while you were growing up)?”

(5) Spirituality, as measured by the question “How spiritual do you consider yourself to be?” Interestingly enough, the correlation of this variable with the PHQ9 depression score was VERY weak (r=0.02) in itself, it became strong only when controlling for the others. This is a weird exception, as all the other variables mentioned here have strong correlations with depression, whether measured on their own, or among controlled variables. 

(6) Low Levels of Conscientiousness, as measured by these two statements: “I see myself as disorganized, careless” or “I see myself as dependable, self-disciplined.”

NOTE: I did not include the Big 5 trait “emotional stability” a.k.a. “neuroticism” in the regression, since it contains aspects of depression already in its questions.

So this suggests that depression is linked to introversion, poor sleep quality, under-sleeping, poor treatment from caregivers growing up, spirituality, and low conscientiousness. Remember though that in each case it could be that it causes depression, is caused by depression, or is caused by something that also causes depression.

The predictive model I used was Lasso regression, with cross-validation, since I expected many of the variables to have almost no relationship to depression (i.e., the results would be sparse). The training set had 696 points, and the test set had 174 points. The training set error was: 54.7% of variance remaining, and the test set (i.e., out of sample error on new data) was 57.5% of variance remaining, so the results appear to be statistically robust (i.e., there appears to be very little overfitting).

Even medical experts in the U.S. and Europe can’t seem to agree about the benefits and costs. As one fairly recent paper put it:

The American Academy of Pediatrics recently released its new Technical Report and Policy Statement on male circumcision, concluding that current evidence indicates that the health benefits of newborn male circumcision outweigh the risks. The technical report is based on the scrutiny of a large number of complex scientific articles. Therefore, while striving for objectivity, the conclusions drawn by the 8 task force members reflect what these individual physicians perceived as trustworthy evidence. Seen from the outside, cultural bias reflecting the normality of nontherapeutic male circumcision in the United States seems obvious, and the report’s conclusions are different from those reached by physicians in other parts of the Western world, including Europe, Canada, and Australia. In this commentary, a different view is presented by non–US-based physicians and representatives of general medical associations and societies for pediatrics, pediatric surgery, and pediatric urology in Northern Europe. To these authors, only 1 of the arguments put forward by the American Academy of Pediatrics has some theoretical relevance in relation to infant male circumcision; namely, the possible protection against urinary tract infections in infant boys, which can easily be treated with antibiotics without tissue loss. The other claimed health benefits, including protection against HIV/AIDS, genital herpes, genital warts, and penile cancer, are questionable, weak, and likely to have little public health relevance in a Western context, and they do not represent compelling reasons for surgery before boys are old enough to decide for themselves.

When experts disagree we’re in a bind. We have little choice but to fall back on our own thinking and research to separate bias from the truth. There are, of course, many people who circumcise their children for religious reasons. For those who have no religious reason, is there any reason to do it?

It seems to me  that we can  formulate a pretty strong argument about circumcision, before we even start to dig into the evidence:  circumcision should only be performed routinely on all male infants if we can identify strong benefits to the child from doing so. That is, without a strong reason to perform the procedure, we should not do it. I conclude this for three reasons:

1. Some of the costs of circumcision are obvious to everyone and very real (e.g. the monetary cost to parents and insurance companies, the pain inflicted on the baby, and the very occasional surgical error, maiming and death). Performing the procedure only makes sense if there are compelling positives that outweigh these well-known costs.
2. The foreskin is a natural part of the human body, and therefore very likely promotes a useful survival or mating purpose. There have been different proposals for what this purpose might be (like keeping in moisture, protecting the penis, or increased sexual pleasure). But one can reasonably assume that foreskin is not merely a fluke, and therefore may well have some use we care about, even if we don’t know quite what that use is.  Because of this, it seems we should not remove the foreskin unless there is a pretty useful reason to do so.
3. Since adults can elect to get circumcised, we should be careful about forcing children to get this procedure at ages where they are too young to make such a choice. If a strong general reason to circumcise children is not found, it would make sense to let each person choose whether they want that procedure when they are old enough to make such a choice.

This line of reasoning leads us to ask: is there compelling evidence that circumcision adds significant value in the form of increased health, improved sexual function, or reduced disease transmission?

Well, you can take a look at the claims and evidence yourself. Here is a table I compiled of the many, many alleged pros and alleged cons of circumcision that I’ve heard made by people on the different sides of the debate. For each claim, I try to link to some studies that support or deny that claim. I focus on meta-analyses of randomized controlled trials, and randomized controlled trials themselves, since they provide far stronger evidence than other study designs. This does not provide a comprehensive set of all studies on male circumcision of course, nor is it a formal systematic review of the literature. But some conclusions quickly emerge:

• Some of the alleged pros and cons that people throw around have randomized controlled trials contradicting them, but continue to be used to support agendas anyway.
• A number of studies are themselves contradicted by other studies (for instance, various results about the impact of circumcision on sexual pleasure seem to point in opposite directions).
• Circumcision does seem to substantially reduce the rates of transmission of HIV from women to men in countries where HIV prevalence is very high.
• Even so, it’s not obvious that this HIV reduction effect is worth it for those who practice safe sex in areas of the world where HIV prevalence is low.
• There is some (small) amount of evidence that hints that male circumcision may make woman more at risk for HIV, but without more studies it’s hard to say.
• Many of the studies used to support claims on both sides are of low quality (for instance, a lot of the evidence of reduced urinary tract infections for circumcised infants is based on observational studies, which are very bad for answering this sort of question…we need randomized controlled trials).
• This topic is extremely complicated! There are tons of different claims being made and there are inconsistent research results for some of the claims. In many cases, few studies have ever been done in the first place, and even fewer high quality studies have been done (though keep in mind that this table is not anywhere close to a complete listing of all circumcision studies).

Click here for the complete table with studies that support and deny each claim.

What causes journalistic distortion?

1. Equal time to each side. There are many issues for which there are two or more reasonable positions that a person can hold. Then there are those issues where one side is supported by nearly everyone who has relevant expertise, and only a few fringe people oppose that view. The trouble is that stories about highly unbalanced issues can lead to a false impression of balance, either because the journalist feels compelled to spend equal time discussing each view-point, or because the journalist is himself unaware of which side is more trustworthy. And a person with highly unrepresentative but highly quotable opinions may be quoted in the article more than is warranted. It may seem less biased to present both sides, but when one side is almost certainly right, an equal presentation may distort more than it informs.

2. Selective reporting. Since news organizations are in the business of selling the news (or, at least, driving traffic to their websites) they have a monetary incentive to produce news that people will be eager to read. Feel-good stories about a dog saving someone’s life can beat out information that might be more important or relevant to most people. What’s problematic from a reality distortion perspective though is that the rate at which events occur and the rate at which they are reported are massively out of sync. For each story about someone coming home from work only to be murdered by their ex-boyfriend, we never hear the millions of tales of people coming home to work only to sit down and eat dinner. This is problematic because the way the human brain tries to estimate how likely something is to occur involves an attempt to retrieve instances of that thing in memory. The more easily you can retrieve those instances, the more frequent you will tend to assume that thing is. If you’ve recently read about a few murder cases, you may have the impression that murder has become more common than it used to be, even if this is merely an artifact of journalists choosing (for whatever reason) to report on more murders. If you can easily think of an example of a shark attack, you may overestimate the frequency of sharks killing people.

The vividness of the accounts we hear can also alter our perception. A vivid retelling not only increases the chance that we remember an account, but also tends to increase our emotional response to it. If you’ve recently read an article that described a gruesome murder in horrid detail, you may subsequently be more afraid when walking alone on an empty street. Through this mechanism, news reading can cause people to have excessive fear of things that aren’t very likely to harm them, and fail to fear far more dangerous things that are rarely reported on. You’re much more likely to die in a car crash than be killed by terrorism, yet in a world where terrorism is reported on constantly, you will likely fear terrorism more.

3. Mix-ups of correlation and causation. Just because X tends to occur together with Y doesn’t mean that X causes Y. In fact, it could be that Y causes X instead, or that both X and Y are caused by some third thing. Unfortunately, reporters frequently get this wrong (or at least fail to make the distinction clear to the reader), especially when reporting on scientific findings. Articles will insinuate that since the latest study found higher wine/broccoli/nicotine consumption was associated with greater longevity/health/focus, that means that wine/broccoli/nicotine must actually cause those benefits. A related problem that you’ll sometimes see (especially in articles about finance) is the implication that since Y came after X, that means that Y was caused by X. It may be true that many stock market investors reacted negatively to a new bill that was just signed into law, but that doesn’t mean that’s a causal explanation of why the stock market fell 1% today. There surely were many factors influencing the change in the market’s price, some tending to push it up, others tending to push it down. Even if it were true that the signing of the bill had a strong effect (which it might be difficult to confirm), that event certainly cannot take all the credit for determining the change in the market’s price.

4. Use of low quality studies. Just because a study “proves” something, doesn’t make it so. In fact, most studies that are conducted are of poor quality for one reason or another. This could be due to a small number of study participants, lack of a control group, lack of randomization, the wrong choice of statistical test, flawed experimental protocol, poor choice of outcome measures, selective reporting of study results, or a variety of other reasons. Unfortunately, journalists rarely make it clear whether a study was of high quality, being mainly interested in what the study claimed to have found. Even the reporting of high quality studies can be problematic, if the journalist fails to mention other high quality studies that found different results. Given all the things that can go wrong in the design and execution of a study, we should be hesitant to accept the results even of those studies that look to be of high quality, until we have seen replication of the study by a different research team.

5. Lack of understanding. Many journalists write about a wide range of subjects. It is rare that they are true experts in the subject of a particular article. But as non-experts writing about what are sometimes very complicated subjects, there is the danger that journalists misunderstand the underlying subject matter. This problem occurs especially often in articles about highly technical research. The issue is compounded further by the fact that journalists are often working under tight deadlines, and so may lack the ability to do extensive background research.

6. Selective use of the facts. Even within a single story, the problem of selective reporting can be substantial. Not all facts in a case are equally entertaining or fit the narrative equally well. There is some incentive to favor those facts that improve the story over drier, though perhaps important material. Of course a political or other agenda on the part of the reporter can also determine which facts he chooses to report on. Since there is a tendency for liberals to read liberal news sources while conservatives read conservative sources, both groups may have their pre-existing views bolstered by selectively reported evidence.

7. Exaggeration of importance. News sells better if it sounds important, so news organizations have an incentive to make their news fit this criteria. One way to do this is to report on stories that actually matter to a lot of people, but sometimes it is better for the organization to just make whatever they’re reporting on sound more important than it is. The next big scientific breakthrough reported on turns out to be completely forgotten a few years or months later (but who remembers?) One of the most common forms of exaggeration in journalism is when a trend is constructed from a few data points. If a handful of celebrities are eating a lot of coconut, or museums have recently become a little more popular among people in their twenties, that doesn’t mean there’s a new fad that the world should hear about.

Choose your news sources carefully, because the information you consume determines what you believe about the world. And as incredibly valuable as journalism is, it can distort reality.