Here are 24 different methods you could use to better understand the way humans work. In each case, I use “trying to figure out new things about post-traumatic stress disorder (PTSD)” as an example. What techniques am I leaving out here?

1. Papers: Reading academic studies (e.g., from social psychology, consumer psychology, cognitive psychology, neuroscience, etc.)

Example: reading the newest academic papers on treatments for PTSD.

2. Products: Carefully examining products and apps that have been successful (as well as, potentially, those that have failed) to understand what human needs they were or weren’t satisfying and why they succeeded or failed.

Example: looking at what mental health apps related to PTSD are popular in the app stores.

3. Outliers: Investigating extreme case studies of real-life events (e.g., case studies of people with extreme psychological conditions who may have been reported about in the news or true stories about what actually happened when people were placed into really extreme conditions).

Example: reading about cases of extreme cults and whether or not each cult produced PTSD-like symptoms in its members.

4. Society: Examining the large-scale trends and organization of societies, including potentially human cultures around the world (American culture, traditional cultures, etc.) and how they are similar or different, and which societal structures tend to be created and what the effects of these structures are, etc.

Example: investigating which sort of societies PTSD appears to be much more prevalent in.

5. Conversations: Talking to smart and insightful people about what they think is true of human psychology. You can also show them the models or theories you have so far to get their criticism and suggestions for improving them.

Example: asking really smart (non-experts) who had PTSD what their current theories about PTSD are.

6. Reading: Reading books, blog posts, media articles, etc., containing information from smart or insightful people about psychology (including potentially writing from life coaches, relationship experts, marketers, thoughtful bloggers, user experience designers, game makers, etc.)

Example: seeing what theories bloggers have proposed regarding PTSD that many academics may not have considered.

7. Experts: Reaching out to experts and talking to them to understand their models of psychology (e.g., psychologists, psychiatrists, game developers, etc.). You can also show them the models and theories you have so far to get their criticism and suggestions for improving them.

Example: talking to world-class experts in PTSD.

8. Textbooks: Reading academic textbooks about psychology, cognitive science, etc.

Example: Linehan’s textbook on DBT and thinking about possible applications of DBT in PTSD treatment.

9. Models: Looking at what models or frameworks of psychology others have created (e.g., in social psychology, theoretical cognitive neuroscience, gamification research, persuasion research, etc).

Example: looking at the model of PTSD that is used in Cognitive Behavioral Therapy treatment.

10. Intuition: Our own intuition for what’s true about humans (presumably based on our first-hand experience interacting with others, as well as our first-hand, ingrained experience of just being a human).

Example: We may have the intuition that after experiencing a traumatic event, people will tend to start having frequent disruptive thoughts about that event, even when doing unrelated things.

11. Interviews: talking to individuals or groups about their thought processes, feelings, behaviors, etc., to understand their individual psychology and then attempting to extrapolate that knowledge to humans more generally.

Example: interviewing 20 people with PTSD about what their experience of it is like and what does or does not seem to help them.

12. Application: attempting to put into practice a psychological theory (e.g., by coaching individuals using a psychological technique or by trying to build an app that implements it) and paying attention to the ways it seems to work or not work.

Example: creating a tool for people with PTSD and making it publicly available, then monitoring how people respond to it.

13. Anecdotes: psychologically surprising or interesting anecdotes that we’re confident actually happened (e.g., because they happened to us or were reported on by a trustworthy source) that may have a bearing on human psychology.

Example: a story a friend told you about how they (believe) they fully cured their own extreme PTSD in a matter of days.

14. Self-observation: careful (and honest) observation of our own thoughts/beliefs/behavior/emotions and internal processes.

Example: observing the mental processes in your own mind that seem to occur after a very upsetting event has happened.

15. Self-experimentation: simply trying techniques and paying close attention to what effects they appear to have and what it feels like to apply them.

Example: try the technique of writing down for a week every upsetting thought that you notice yourself having, as well as writing down everything you observe or learn about the experience of doing this.

16. Deduction: combining propositions we already believe to produce new propositions we didn’t realize before that may be true.

Example: knowing that women with PTSD are much more likely to have associated depression than men with PTSD, and knowing that depression is a strong risk factor for attempting suicide, we might predict that women with PTSD are more likely to attempt suicide than men with PTSD (even though they may not be more likely than men to actually commit suicide).

17. Experiments: Conduct your own online randomized controlled trials, surveys, longitudinal studies, or fMRI studies, etc.

Example: looking at which variables that don’t seem obviously PTSD related are, in fact, strong predictors of whether someone has PTSD.

18. Statistics: looking up known statistics about a phenomenon.

Example: looking up whether younger people or older people are known to be more likely to get PTSD in any given 12-month period.

19. Data: finding existing data sets (e.g., large longitudinal surveys or government data sets) and running your own statistical analyses to test hypotheses.

Example: measuring whether it is true that after someone gets a pet, they are more likely to recover from PTSD than similar people who do not get a pet. Running studies can also be a good way to search for new hypotheses. For instance, examining which of many variables in the data are most associated with a rapid recovery from PTSD.

20. Literature: looking at portrayals of psychology in literature, art, myths, stories, philosophical writings, etc.

Example: What hypotheses do Kafka’s novels give us about the nature of trauma?

21. Crowdsourcing: Request suggestions for hypotheses on a social media platform like Facebook or X or on a question-answer site like Quora.

Example: posting to Facebook asking people to suggest theories for why PTSD sometimes does and sometimes doesn’t happen after extreme trauma.

22. Social experiments: bringing a group of people together under a new set of chosen social rules or guidelines and observing what happens.

Example: getting a small group of people together with the idea that it will be a time to discuss traumas that members of the group have experienced.

23. Evolution: considering what possible evolutionary function different psychological phenomena might have.

Example: consider ways that aspects of PTSD responses might be evolutionarily adaptive.

24. Practitioners: examine how people “in the field” who benefit from being able to understand this aspect of psychology do their work.

For example, at veterans hospitals where rates of PTSD are high, investigate what techniques and approaches the therapists have developed to work successfully with their patients.

This piece was first written on November 16, 2017, and first appeared on my website on July 22, 2025.

After noticing a number of times that people’s feelings about formal credentials can differ dramatically and that this seems to impact their views on certain important topics, I’ve been working on defining a “credentialist” trait (or attitude).

In a nutshell, the non-credentialist/credentialist spectrum, as I’m defining it, captures how important a person thinks formal credentials are, as well as how they feel those credentials should influence who we should trust and who should express opinions (e.g., should only formalized experts comment on a topic, or is it good for non-experts to comment as well?)

I developed a 4-minute test to measure the trait, so if you’d like to find out if you are a “credentialist” or “non-credentialist” (which I define as being in the top or bottom 20th percentile of each trait) or find out your own credentialist score, you can take the test here:

https://programs.clearerthinking.org/credentialist_test.html

Here are simplified/extreme prototypes to illustrate the distinction (few people are as extreme as these prototypes):

*Credentialists* get annoyed when someone without the right credential is giving their opinion on a topic, are impressed by formal degrees (e.g. PhDs and MDs), do not like it when non-experts have their own personal theory about a topic, trust people a lot more when they have formal credentials, think its unlikely someone could get really good at a complex topic without formal training, think that non-experts should not contradict experts, would go to school to learn a topic if they wanted to get good at it, find it annoying if startup founders talk about disrupting industries they have not already worked in, tend to describe people in terms of their schooling and job history (rather than, for example, their personality), and trust the opinion’s of people a lot more if they went to an excellent college.

*Non-credentialists* think that it fine (or even good) to express opinions when you’re a non-expert, are not particularly impressed by formal degrees, don’t view degrees or certifications as a strong indicator of trust, think it’s fine (or even good) for non-experts to criticize the views of experts, are prone to teach themselves material rather than going to school for it, don’t mind startup founders attempting to disrupt industries from the outside, tend not to describe people in terms of their schooling and job history, and don’t view the quality of the college a person went to as a significant factor in whether to trust their opinions.

I measured the trait on a 0 to 1 scale, and on the 143 people I collected it for, it has a mean of 0.55 and a standard deviation of 0.16, with a pretty nice bell curve shape.

Interestingly, I found a very low correlation between credentialist scores and education [r=0.03], identifying as female [r=-0.04], and income [r=0.03], and little correlation with age [r=-.08]. This suggests that credentialist differences have little to do with demographic characteristics!

Furthermore, whether you are a credentialist does not even seem to have that much to do with whether you yourself have credentials, as responses to the question “I myself have substantial formal credentials” had a correlation of only about r=0.13 with the credentialist scores. It also has little to do with how ambitious people are, as responses to “I have highly ambitious goals for what I will achieve in my life” had a correlation of only r=0.07 to credentialist scores.

Of the 22 questions I tested people’s agreement on in order to measure the trait, the 2 most effective questions (in the sense that they correlate highly with the average of the other questions but don’t correlate very highly with each other) are:

Q1: It annoys me when someone without the right credential is giving their opinion on a topic (e.g., a non-doctor commenting about medicine or a non-accountant commenting on accounting)

[r=0.71 against the average of the other 21 questions]

and

Q2: I am very impressed by formal degrees (e.g., PhDs, MDs, JDs, etc.)

[r=0.54 against the average of the other 21 questions, yet a fairly low r=0.24 with respect to Q1]

The 22 questions I developed hang together nicely and point in generally the same direction. Basic factor analysis revealed only one main factor in the questions, and the question least correlated to the others still had a positive correlation of r=0.34 with the average of the other questions (which was “If I heard that someone had won a prize in their field, I would think very highly of it”).

Most people fall in the middle of this trait, of course (e.g., viewing credentials as at least somewhat positive but the lack of them not that negative), without an extreme viewpoint either way. However, here are some anonymized qualitative responses I collected from people at the tail ends of the spectrum:

Credentialists:

“I don’t respect people who don’t have formal credentials, and one of my biggest pet peeves is people–even really intelligent people–speaking about things they aren’t experts on. Just because someone is known, or even renowned in another field, doesn’t place their opinion on another topic anywhere above another non-educated person.”

“People with formal credentials have generally gone through a rigorous peer review process that demands a considerable depth of understanding and knowledge.”

“People who don’t have credentials have no business talking about things they know nothing about. That’s how misinformation gets spread, and misinformation is harmful to society as a whole.”

Non-credentialists:

“I think that people can have a valid voice no matter their level of formal schooling. The opposite also holds true: a degree is not necessarily representative of ability.”

“I feel that colleges have become scams. They are money sponges that make you pay for your own brainwashing. I value intelligence and how well-read someone is on a subject, and we don’t need a self-appointed team of left-wing experts to ‘allow’ us to do that for ourselves anymore. People with credentials have more money than sense and have been taught what NOT to think more than they have been taught HOW to think.”

“While I respect those who have worked to earn professional credentials, people who are self-taught oftentimes know much more about a subject than someone with an expensive degree.”

Note: designing this scale and writing this post is a very non-credentialist thing to do since I’m not a social scientist.

This piece was first written on July 21, 2017, and first appeared on my website on June 10, 2025.

Given the incredibly high levels of disagreement in these areas, and the poor quality of studies, it often seems like we know almost nothing.

Below is my attempt (via a combination of brainstorming and crowdsourcing) to list what there does seem to be a high rate of consensus on. Please let me know if you notice any mistakes.

Tentative List of Universally Recommended Health Interventions

(1) Don’t consume a lot of sugar (at best, it’s “empty calories” and could lead to tooth decay. But some claim it is much worse than that).

(2) Exercise regularly – it’s best to rotate which type of exercise you do, being very careful to avoid injury, especially when you are getting into new forms of exercise. (It is less clear what forms of exercise are best for long term health, e.g., strength training vs. cardio, and it’s less clear quite how much exercise you should get – also, extremely high levels of exercise are believed to be associated with increasing some health risks, for instance doing Iron Man triathlons).

(3) If you are going to eat a lot of carbohydrates, generally you should choose complex carbs over simple carbs (usually whole grains are also recommended over refined grains, but some argue that whole grains should be sprouted/soaked to remove parts of the seed that are designed to protect it from digestion.)

(4) Brush your teeth with a fluoride toothpaste at least once per day (though perhaps it is not actually a good idea immediately after eating, especially if you’ve been eating acidic foods). The suggestion is to brush before breakfast or wait an hour after eating, and beware of brushing too often or too vigorously. Brushing twice per day may be better than brushing once. Also, note that a VERY small percentage of healthcare professionals are anti-fluoride.

(5) Hydrate regularly throughout each day, as soon as you feel thirsty, but before you are likely to feel thirsty as well (since thirst indicates that you may already be dehydrated, so you want to pre-empt feeling thirsty). Hydrating with water is the safest bet, though it’s not clear how much liquid you need in total (the 8 cups of water a day theory seems to be bullshit). It also is unclear whether it’s important to do this with water, or if other drinks, like unsweetened tea, are acceptable replacements. Hydration may also be good for your teeth by reducing dry mouth – some argue that even mild dehydration can negatively impact mood and performance.

(6) Eat plenty of vegetables (preferably not deep-fried ones – note also that there do exist a very small number of people in the health field who advocate an essentially zero-carb or meat only diet).

(7) Don’t eat a lot of deep-fried foods in general.

(8) Take Vitamin D3 supplements if you are >60 years old and don’t get a lot of outdoor time, or if you are in the general population, take them if you get very little sunlight.

(9) Avoid frequently drinking large quantities of alcohol.

(10) Avoid frequently consuming tobacco products (but since many of them are addictive, that means it’s safest to avoid them altogether).

(11) If you have the ability to make yourself lose weight and keep it off, you should try to lose weight/reduce body fat if you have a very high body fat percentage, or if you have a lot of body fat around the gut area (the extent to which mild to moderate obesity is bad per se is somewhat debated, as in some studies mild levels of obesity were sometimes even correlated with better health outcomes) – avoiding metabolic syndrome and poor blood sugar dysregulation may be more necessary than avoiding a very high body fat percentage though the two are significantly correlated – note also that lots of data suggests that three years after a diet most people have regained the weight they lost, and some say that regularly cycling your weight by losing then gaining then losing again could be unhealthy (so the value of a blanket “lose weight when obese” recommendation is questionable).

(12) If you are going to eat something sweet, a piece of fruit is a better bet than candy or sugary baked goods.

(13) Avoid consuming trans fats.

(14) Don’t consume excessive amounts of mercury (which is found in many fish – some say that tuna, king mackerel, marlin, orange roughy, shark, and swordfish are particularly worrisome).

(15) Don’t eat a lot of foods that are burned to the point of being severely blackened (at best burning tastes bad and destroys the nutrients in food, at worst, it might increase the chance of getting cancer, it’s unclear).

(16) Spend some time outdoors in the sun each week.

(17) Avoid getting frequently sunburned (especially avoid severe sunburn, e.g., blistering).

(18) If you are unusually low in any vitamin then you should consume more of it (but if you have relatively normal levels, there is not a consensus on whether you should have more of any vitamin, except perhaps Vitamin D for the elderly which seems to be basically agreed upon – there is also disagreement about whether vitamin pills are as effective as vitamins from whole foods, or in which cases you should take sublingual dissolving vitamins or vitamin shots instead of pills).

(19) If you are a strict vegan, take vitamin B12 supplements (while only a very small amount of B12 is needed by humans, it is not reliably found in plant-based foods).

(20) Don’t get addicted to any drugs (prescription or non-prescription) other than perhaps caffeine.

(21) Don’t run an increasingly large sleep deficit (though the amount of sleep each person requires to avoid a deficit seems to vary considerably).

(22) Wash your hands with soap regularly (though some claim that special anti-bacterial hand soap is not a good idea)

(23) Don’t regularly have non-negligible amounts of caffeine within a few hours of going to bed (though people’s sensitivity to this seems to vary a lot – some people may be able to get away with it, but on average it impairs sleep quality).

(24) Avoid long stretches of inactivity (i.e., take breaks where you move around).

(25) When typing, avoid having your wrists bent at a significant angle for long periods, avoid having to bend your neck substantially downward or upward to see your computer screen, and avoid sitting with your back hunched forward for long periods (some people find that using a vertical mouse helps if you have wrist pain, and some people report benefits from using a laptop stand to raise your laptop to a more reasonable height).

(26) Each week spend at least a bit of social time with people you get along well with.

(27) If you have very high levels of anxiety, depression, or hopelessness, you should seek treatment as soon as possible (e.g., you could try Cognitive Behavioral Therapy with a psychologist, go to a psychiatrist to try antidepressants, or take up a Mindfulness-Based Stress Reduction practice).

(28) Eating a diverse range of healthy foods is usually better than eating a narrow range of foods (of course, a diverse range of unhealthy foods is still unhealthy).

(29) Avoid very high doses of certain vitamin and mineral supplements where overdoses are known to happen (e.g., Iron supplements, vitamin A and vitamin B-6 – beware of mega-dose vitamins in general unless you know what you’re doing, as they are unlikely to be helpful and could be harmful).

(30) If you have a broken bone or reasonable sized cut or scrape that appears as if it could be infected, go to a doctor immediately (some broken bones require splinting to heal properly, infected wounds may become dangerous without proper treatment).

(31) If you have a mole that violates the ABCDE rule, get it checked out by a dermatologist immediately. That means you should get it checked out if it has: Asymmetry (if one side of the mole doesn’t match the other), Border irregularity, color that is not uniform, Diameter more than 6 mm (which is about the size of a pencil eraser), and Evolving size, shape or color.

(32) Highly processed meats (e.g., hot dog or bologna) are worse for you than less processed ones.

(33) Do things to keep your brain active, such as learning something new each week or doing something that is mentally taxing.

(34) If you have high levels of stress, try to reduce them using whatever techniques you find effective (high stress has been linked to various negative indicators in the body – techniques that some people find effective to reduce stress include Cognitive Behavioral Therapy (CBT), meditation, progressive muscle relaxation, yoga, walking in nature, cuddling with a pet, or exercise that keeps your heart rate elevated for a reasonably long period).

(35) Keep your sleep cycles at least roughly in sync with the dark/light cycle of the planet (i.e., do most of your sleeping at night, and live most of your waking hours during the day).

(36) Don’t regularly drink alcohol before going to bed (as it is known to cause poor sleep).

(37) If you think you may be suicidal, or you have made plans for how you might commit suicide, you should call a suicide hotline immediately, and afterward make an appointment with a therapist or psychiatrist to see them as soon as possible (and tell this person about your suicidal thoughts).

(38) Keep your living environment at a comfortable temperature, generally in the 65-75°F (18-24°C) range.

(39) If you’ve had pain in your mouth or teeth for more than three weeks, go to a dentist!

(40) Wear your seatbelt every time you are in a car. (Even if it’s a short ride, do it anyway, because, if you don’t, you are eroding the seatbelt wearing habit – also, a note for New Yorkers that even car accidents in taxis can hurt you badly when your face hits the glass in front of you.)

(41) Folic acid supplements for women who are or may be pregnant (to help avoid birth defects).

(42) Either keep health insurance, or live in a country where health care is universal, or keep enough savings that in a medical emergency, you can get the treatment you need.

(43) Don’t drive when drunk or high on drugs, or be in a car driven by such a person.

(44) Don’t send text messages or do other distracting things while you’re driving, or be in a car driven by someone who is distracted.

(45) If you are depressed, bi-polar, or potentially suicidal, don’t live in a house that contains a gun (or other easy means of suicide).

(46) Always use condoms when having sex with new partners if they have not been recently screened for STIs.

(47) Always use some form of birth control when having heterosexual intercourse unless you are planning a pregnancy.

(48) Women should avoid wearing tampons for more than 8 hours at a time to reduce the risk of Toxic Shock Syndrome. (Some say that the lower the absorbency of the tampon the lower the risk)

(49) If you have a severe bacterial infection, take antibiotics. (Note that the type of bacteria and antibiotic matters.)

(50) If you have diarrhea, continually drink oral rehydration solution (however if you have severe dehydration or are vomiting, seek medical treatment – you can make your own oral rehydration solution using 6 ‘level’ teaspoons/25.2 grams of sugar and 0.5 teaspoon/2.1 grams of salt in 1 liter of water – or you can drink Pedialyte – diarrhea is actually surprisingly dangerous, so stay hydrated!)

(51) If you are taking antibiotics, and you are not immunocompromised, then take a probiotic with the antibiotic to reduce the chance of antibiotic-induced diarrhea. (Note that there is a lack of consensus on the amount and type of probiotic to take).

(52) Eat foods containing fiber. (Sometimes about 30g per every 2000 dietary calories is recommended, but recommendations vary)

(53) Don’t live or work in a place where people regularly smoke indoors.

(54) If you are a women who is planning on having children, try to have them before you reach age 40 (the younger you do it, the safer it is for you and the baby, though with advances in early screening many problems associated with pregnancy at an older age are now caught – fertility also increasingly becomes a problem with age).

(55) Avoid exposure to asbestos and lead paint.

(56) If you live in an area with disease-carrying mosquitos, use a mosquito net on your bed when you sleep.

(57) If you have sudden and severe symptoms of illness that do not resemble a cold or flu, or unexplained symptoms of illness that continue for a long time, go to a doctor.

What else should be on this list that I missed? What am I mistaken about that I should remove from this list (because there is actually some disagreement among experts)?

Also, here are some other health and nutrition questions that didn’t make this list because, while many advocate strongly for one side, there still seems to be a reasonable amount of disagreement (rightly or wrongly):

Important Health Questions Experts Don’t Seem To Have Consensus On

(a) How bad is saturated fat, if at all?

(b) How useful is omega 3 supplementation, if at all?

(c) How bad are carbs versus other macronutrients?

(d) Is there any benefit (or harm) to getting more than the RDA of protein (0.36 grams per pound of body weight)?

(e) Is flossing truly effective? (If you like flossing, or at least don’t mind it, it may well be worth it, but the benefits are not as clearly established as one would ideally like, and there have been some claims, possibly false, that it can cause bacteria to escape from your mouth into your body in a way that could be bad)

(f) Does dietary cholesterol lead to high blood cholesterol? (apparently, the FDA just released new guidelines on this that say “no.”)

(g) Is blood cholesterol correlated enough with bad outcomes that we should care about it, per se?

(h) What types of preventative screening/testing should everyone routinely get?

(i) What dietary supplements (if any) should a healthy person take?

(j) Is there any harm from Aspartame or other artificial sweeteners like Splenda? (all of the many randomized controlled trials on Aspartame on humans I’ve looked out found no negative effects except headaches in a small subset of people, but other studies in rats show weird effects that are hard to interpret, and a lot of people are anti-Aspartame without providing clear reasons – note that sugar alcohols like Sorbitol cause temporary digestive problems for some people.)

(k) Is polyunsaturated fat good for you, bad, or neutral?

(l) How bad is meat for you as a broad category, or is it too broad a category to generalize?

(m) How much should you limit your salt intake, and do moderate levels of salt intake actually increase blood pressure (outside of the rare group of “salt-sensitive” people)?

(n) Which type of cooking oil (e.g., olive oil, coconut oil, avocado oil, etc.) should you use or avoid?

(o) How much exercise is ideal, and of what forms? [as Stuart Buck points out: “If the goal is to be strong and build muscle mass, lift weights. If the goal is to be fast, agile, etc., work out more like a sprinter and/or a professional basketball player. If the goal is aerobic fitness and endurance, do lots of long runs or swims, etc. But for long-term goals like longevity, that’s where we don’t have rigorous evidence (too much selection bias)”]

(p) Is going into ketosis (by lowering your carb intake dramatically) a good or bad idea? (Many doctors think it is not a healthy state to stay in for a long time, but anecdotally some people who do this diet swear by it)

(q) Is intermittent fasting a good idea?

(r) Is it important to go to bed/wake up at the same time every day?

(s) Is there any real benefit to eating organic foods?

(t) Are “grass-fed” animal products healthier than non-grass fed ones?

(u) Is there any real difference (in your body) between sugar and high fructose corn syrup? (common sense about chemical composition and some studies suggest the answer is that there is no difference, but many people think high fructose corn syrup is worse)

(v) What’s the optimal mix of macronutrients?

(w) Do metformin, rapamycin, resveratrol, human growth hormone, blood transfusions from young people, basis (nicotinamide riboside and pterostilbene), or other NAD boosters increase lifespan for healthy individuals?

(x) Are GMO foods actually risky, or are they fine?

(y ) Is the heuristic of eating “natural” or “whole” foods actually accurate, or does it exclude too much?

(z) Are probiotics (like acidophilus) useful to take for a generally healthy person?

(aa) How important is stretching, what type of stretching (static vs. active) is best, and when should you do it (just before you exercise, just after, or at other times)?

(ab) How bad are pesticides on our foods (which types are bad, and how much of them do we have to consume before problems begin)?

(ac) Should you take a multivitamin pill? (the tide has been turning against them as repeated studies fail to find a benefit in healthy people, but some experts still recommend them – also for water-soluble vitamins, we seem to pee most of it out when we take large doses)

(ad) Are there vitamin/mineral deficiencies that a significant proportion of people in developed countries have? (e.g., possibly magnesium, potassium, choline, D3, K2)

(ae) Is it helpful to wake up when the sun rises each morning?

(af) Do heavily calorie-restricted diets improve longevity in humans (like they do in mice)?

(ag) Is using mouth wash (like Listerine) good for your oral health?

(ah) Is donating blood (for men and women who don’t menstruate) good for health?

(ai) Does zinc or echinacea reduce the severity or duration of colds when taken daily or when taken at the onset of symptoms?

(aj) Does the source of calories (e.g., from carbs, fat, or protein, with fiber or without) matter significantly when it comes to non-water body weight change (or does a calorie have the same effect regardless of the source when we are talking just about weight change and excluding water weight)?

• You know that “something” exists.
• If you have mental experiences, then you know that “you” exist (though coming up with a reasonable definition for what “you” means can be remarkably tricky).
• If your mental experiences are varied, then you know that whatever exists creates varied mental experiences.

With some cleverness, you may be able to add to this list a few more things that you know with total certainty, but not many. In fact, almost everything that we think we know we cannot be completely sure about. For instance:

• You take it for granted that five minutes from now you will still be alive, but there is a non-zero probability that your heart will give out before then.
• You feel as though the world you see around you is reality, and yet, that is also what it would feel like if you were in fact in an extremely detailed simulation of reality (for instance, your brain could be hooked up to some sort of computer). So you cannot know for certain that you are not now living in a simulation rather than having experiences based on physical reality itself.
• You cannot know, with total certainty, that you are not dreaming right now.
• You cannot know, absolutely, that you have not gone mad. Many things that you think are real could be delusions. Of course, you’d expect your delusions to feel totally correct to you, and not seem delusional at all.

Virtually everything involves some amount of uncertainty (even the best predictions will contain some error), and many things are highly uncertain (in the sense that our attempts to model them lead to uncertain predictions). Consider our uncertainty about processes on different physical scales.

On a large scale there are weather systems and economies, which depend on very large numbers of variables that are difficult to measure accurately. Weather depends on temperatures, pressures, geography, sun output, cloud cover, pollution, bodies of water, etc. Economies are impacted by spending behavior, regulation, war, technology, disease, culture, and so forth. What makes large systems especially hard to predict is that many of them are chaotic in nature, so small changes in the values of the variables by which they are influenced can lead to large changes down the line. For instance, a relatively small shift in the pressure and temperature in one area of the world may eventually lead to a hurricane hitting a far away area. Or, in an economy, a piece of legislation passed today that deregulates certain company behaviors could lead to increased use of leverage, which could result in a market collapse a decade later. Political events are another good example of large-scale uncertainty, because they tend to be highly complex and chaotic. That means they should be hard to predict. In fact, the empirical data says that most political forecasting “experts” are not significantly more accurate than random guessing or a non-expert predictor, though some fairly small fraction of such experts do seem to have meaningful predictive power.

On a medium scale, there are humans and roughly human-sized objects. Objects on a human scale, like balls and bookcases, tend to be quite each to predict, in part because they are accurately modeled by the relatively straightforward laws of Newtonian Mechanics. If you throw a ball you have a decent idea of where it is going to go, though the details of its trajectory will still be uncertain. If you drop a bookcase off of the Empire State Building, you know it will fall downward, and can roughly estimate when it will hit the ground, though exactly where it will land will depend on unpredictable factors involving air flow across its surface. However, predicting the location of the cracks along which it will shatter when it finally hits the ground is far beyond our ability to forecast.

Unlike balls, which are fairly straightforward, human minds are very unpredictable. The problem is three-fold: the complexity of the brain, our lack of knowledge about brains in general, and the inaccessibility of the present state of any particular brain. It can be hard enough to tell if someone you’re talking to is annoyed, engaged in a pleasant day-dream, or feeling awkward. But predicting what they will say next will be even more difficult. Brains can even act as uncertainty generators. You can give a man a physical object, like a book, which can lead him to believe some idea that he then feels compelled to spread. If he is successful enough in his proselytizing, he may spread this idea to hundreds of others, who themselves spread the idea to tens of thousands more. The large-scale fate of the world may be altered due to one particular person having seen one particular book, a result that was dependent on the great complexity of the human brain.

Knowledge of psychological studies does, of course, allow us to predict some things about human behavior. For instance, in Milgram’s famous experiment, it was found that if a person who seems sufficiently authoritative insists that a test subject administer high voltage shocks to another person, even though it seems obviously dangerous to do so, about two-thirds of the test subjects will comply. The trouble is that even robust psychological findings like this one involve much uncertainty. For one thing, it is difficult to predict in advance who the one-third of non-compliers are going to be. Furthermore, the fraction of participants that comply is influenced by a variety of factors, so one cannot assume that this two-thirds number will hold in similar situations. For instance, when the authority was not in the room with the test subject, communicating only by telephone, compliance fell from two-thirds to only about twenty percent. When other test subjects (actually actors who are confederates of the experimenter) refused to comply with the authority, only ten percent were willing to continue with the experiment.

While there are some human behaviors that can be predicted very accurately (if you touch a hot pan, I predict you will pull your hand away, if you are handed a deep-fried oreo, I predict you will eat it, if you attempt a certain extremely difficult logic puzzle, I predict you will not be able to solve it in 5 minutes). But, at this stage of our understanding, much of the human behavior that we care about predicting can only be modeled in a probabilistic fashion, with a lot of uncertainty in our estimates. We are forced to use statements like “most of the time when you put a person in situation X, they will have behavior Y.”

While one might hope that at the small-scale things become certain, this is unfortunately not the case. It is now known that atoms are governed by the laws of quantum mechanics, which involve inherent, fundamental unpredictability. In fact, in quantum mechanics it doesn’t make sense to even talk about the exact position and velocity of a particle. We can only know the probability of a particle being measured to be within a certain distance of a particular location, and having a velocity within a certain range of values. The Heisenberg Uncertainty Principle makes this idea more precise, encapsulating the idea that the more certain we are of the position of a particle, the less certain we must be of its velocity, and vice versa. As systems of particles become larger and larger (forming objects such as carrots and boulders) the uncertainty principle can be essentially ignored, because the size of the quantum uncertainty is very small compared to the size of the object itself.

Uncertainty is inescapable. It manifests in large-scale chaotic systems, like economies and weather systems, in the details of the trajectories of medium-scale objects, in human behavior, and in small-scale systems . For many of the most interesting predictions we would care to make, we cannot avoid making them in terms of probability. Rather than being able to say, “X will happen,” we end up only being able to speak in terms of “the probability that X will happen.”

To know what an expert of a particular type is truly expert at, we need to understand the system that creates that type of expert. What are these experts rewarded for doing or knowing? What actions make them more or less likely to succeed? If in a particular field a behavior X is strongly rewarded with prestige, promotions or wealth (or not doing X is strongly punished) then we would expect many experts in that field to do X. Those that do it naturally or adapt to the system and do it consciously will be more likely to succeed, while those failing to do it will tend to remain obscure or move into other fields. What determines an expert’s actual expertise is not just what her field is ostensibly about, but what knowledge and behaviors she is rewarded or punished for having.

Let’s take the field of pure mathematics as a case study. Experts in this field are created through the University system. Top expertise is determined primarily through the impact and quantity of publications. To produce many publications with high impact, you need traits like an expansive knowledge of theorems and methods, a strong ability to reason logically, the inclination to work for long periods on difficult problems, and sufficient creativity to generate novel ideas. These are the things we typically assume a mathematician has to be good at. But top mathematicians must have another extremely important skill as well: a knack for producing work that other mathematicians will consider important. It is largely other pure mathematicians who determine what gets published, which grants get issued, and what jobs get offered, so it is largely other pure mathematicians that ultimately determine who the top experts in their own field are.

To pure mathematicians the importance of a theorem is generally not determined by whether it has real world applications (in some circles, applications may even be considered a negative). It also is only partially captured by how hard the theorem was to produce. Importance depends on things like whether:

• other pure mathematicians find the theorem aesthetically pleasing (it should be simple yet non-trivial, powerful, and of an appealing form)
• it has implications for or connections to the work of many other pure mathematicians
• it provides new insights related to something that mathematicians care about that was previously not well understood
• other respected mathematicians have tried and failed to produce something similar
• it is part of a mathematical subfield that currently happens to be trendy

Mathematicians who have a non-standard aesthetic (i.e. their definition of what is beautiful or interesting in math differs a lot from what other pure mathematicians think) are at a big disadvantage, as are ones who work in areas that happen to be unsexy at the moment, and ones whose work doesn’t tie nicely into work that many other mathematicians care about. We should expect therefore that top pure mathematicians don’t just have expert mathematical knowledge, but also are expert at playing the pure mathematics game. That means being excellent at producing work that fits certain subjective criteria that are highly valued by many other pure mathematicians.

The pure mathematics game is not a bad game. It is very intellectually stimulating, and it sometimes yields results that end up being extremely useful in practical applications (especially in physics and engineering). But it certainly is not the only game that could be played in mathematics, and its rules are not optimized to make it as beneficial to society as it could be.

The rules of the game are self-perpetuating to a significant extent. Those who play by different rules are less likely to succeed in the field, and those who are good at playing the existing game are more likely to succeed. Those who advance are eventually in a position to decide who else gets to advance, and generally choose people with tastes similar to their own, so the game continues for another round. The details of the game drift over the decades as fashions come and go, and professor die. But certain aesthetic values and notions of what is important can get preserved over long periods.

A similar analysis can be done for nearly any field. What are top doctors expert at? Not just medical knowledge and procedures, but also at satisfying those who reward them. One group that rewards doctors (with their money) is patients. If a doctor is really good it is obviously true that a patient’s condition is more likely to improve, and therefore the patient is more likely to come back next time they have a problem. But it can be very hard to tell whether your improvement was due to your doctor’s skill, or whether you would have gotten better just as fast if you had gone to an average doctor, or even to no doctor at all. Many conditions get better regardless of what treatment is given (and some get worse even when treated properly), so it may be very difficult for patients to accurately assess the true skill of a doctor.

Doctors don’t get rewarded just for making you well fast. They also get rewarded for doing things with you that you believe a good doctor would do, since those behaviors make you view them as a good doctor (and so you’ll be more likely to go back in the future and tell your friends about them). These “good doctor” behaviors might include spending extra time with you, using obscure but impressive sounding medical language, speaking with authority even when they aren’t very sure of themselves, running lots of tests that aren’t really necessary, or using expensive and complex looking machines. There are plenty of ways a doctor could give you the impression of having done a great job that have little or nothing to do with getting you well faster or at low-cost. A doctor who gets you well fast but doesn’t SEEM like a good doctor could in some cases keep fewer patients than a doctor who seems to be better than he truly is. So we should expect to find that top doctors (of the type that rely on patients for almost all of their income) are not just medical experts, but are usually very good at meeting people’s expectations about how a good doctor should behave.

Some exercises for the reader:

• What are art critics truly expert at?
• What are lawyers truly expert at?
• What are contractors truly expert at?

When you consider what experts in a field are truly good at, don’t just ask “what is this field about?” but also, “what traits or behaviors get rewarded or punished in this field?” Behaviors that get rewarded or punished are not necessarily the ones we typically associate with the field, but considering them gives us information about how the experts truly behave.

Influences: Robin Hanson