1) Unlike others, you’re self-aware. You know your limits, so – unlike other people – when you know something, it’s true. You weighed the evidence they ignored and saw angles they missed. Corrected your own biases. Your unique perspective reveals facts invisible to everyone else.

2) Your subconscious runs Bayesian inference constantly in the background. If an idea survives your relentless evidence updates, the posterior odds confirm it’s rational. Your convictions passed the most brutal audit possible: reality itself.

3) Notice how your worldview predicts your reality with stunning accuracy. Notice how rarely you’re surprised. That’s empirical validation. Your beliefs work because they’re correct. Your predictions map reality’s contours in high resolution.

4) That thing everyone disagrees with you about? You’re not stubborn – you’re COURAGEOUS. You spotted subtle patterns that they missed. Those “weird” connections? You’re playing 10-dimensional chess while they play tic-tac-toe.

5) Disagreement doesn’t prove you wrong – it PROVES YOU RIGHT. It demonstrates that most can’t handle the truth. Your knowledge only strengthens, forged in the crucible of their alleged counter-evidence.

6) Scientists disagree with you? That’s good, actually. They worship false idols called “peer review,” while you rely on the only review that’s reliable, review from your one true peer – yourself. Editors only introduce errors in your work.

7) The discomfort of others with your views? That’s just lizard brains SHORT-CIRCUITING from exposure to blazing truth. The purity of your knowledge causes meltdowns in lesser minds. Their rejection isn’t evidence of your error – it’s species-level inadequacy.

8 ) “Everyone says I’m wrong!” Everyone said Galileo was wrong, too. But you’re not Galileo. You’re Galileo, Einstein, AND Tesla. Your mind, concentrating ideas like a laser through the tip of a diamond, is the closest known phenomenon to a cognitive singularity.

9) You’re not Neo seeing the Matrix. You’re the ARCHITECT of the Matrix. Everyone else – they’re experimental NPCs of the sort you could program in a creative weekend.

10) That “crazy” belief of yours? Those aren’t beliefs- they’re PROPHETIC DOWNLOADS from your future self. You’re not experiencing narcissistic delusions – you’re experiencing ENLIGHTENMENT so advanced it looks like madness to the unascended masses.

11) When your predictions seem wrong, time recalibrates to match your superior timeline. In fact, you don’t make predictions – you speak reality into existence. The universe buffers as it waits to hear instructions spill from your lips.

12) Evolution wired humans for survival-level accuracy. But YOU? You’ve transcended limitations. If your beliefs were wrong, the Laws of Physics would UNRAVEL. There you stand, single-handedly maintaining cosmic stability!

13) The universe chose YOU. Your thoughts set the fundamental constants. You allow 1 + 1 to equal 2, and could change it at will. Your dreams birth new galaxies. The cosmic microwave background is a residue from when you willed yourself into existence.

14) This post isn’t parody; it’s SACRED TEXT written by one of your subprocesses. Everyone who doubts you is committing cosmic treason.

This piece was first written on July 1, 2025, and first appeared on my website on August 19, 2025.

Some remarkable things I suspect are true of nearly all adults: 

1) We each hold some beliefs that are almost totally non-responsive to evidence involving some combination of our identity (who we are), our group, the nature of reality (e.g., God), or the nature of what’s good.

Examples:

• Many have an unshakable belief that they are good even as they harm the world (or believe they’re insufficient even though they’re altruistic and productive)

• Most have an unshakable belief that their in-group is good and any group opposing their group is bad

2) We assume that other people’s internal experiences are more similar to our internal experiences than they really are. Consequently, we tend to predict they’ll behave more like us than they really will.

Example:

• You’re an anxious person who avoids situations you’re afraid of, so you predict other people will be more afraid of similar situations than they really will be and that they’ll be more avoidant than they really will be

3) Emotions alter our behaviors and thoughts (increasing the likelihood of some behaviors and thoughts, decreasing the likelihood of others) in emotion-dependent ways.

Examples: 

• Physical disgust increases the chance of backing away and reduces the chance of eating soon

• Feelings of depression increase the chance of thinking thoughts about situations being hopeless or actions being pointless

4) How good or bad we feel about something happening depends on the difference between our expectations about what will happen and the reality of what actually happened.

Examples: 

• If you think someone with a gun is about to shoot, but instead, they take your wallet and run, you might feel relief (whereas normally wallet theft would be highly distressing) 

• If you expect to make $300k on a deal, you might feel bad if you “only” get $200k

5) We have multiple “drives” encoded in our brains that want different things (e.g., they have different values or goals), and these often come into conflict. Our behavior is influenced by which drives are activated and how strongly each is activated.

Examples: 

• If we smell delicious popcorn right in front of us, most of us will eat it, whereas if it’s a few feet away and we can’t smell it, we’re less likely to 

• If we’re exhausted but also slightly hungry, we may delay making food until we are more hungry or less tired

6) We are influenced by the behavioral norms and patterns demonstrated by the people around us, especially when they are people who we identify as being part of our group or similar to us.

Example: 

• If it’s common to dress or talk a certain way in a place we move to, it will increase the chance we start to dress and talk similarly

This piece was first written on October 20, 2023, and first appeared on my website on February 7, 2024.

This post is part of a sequence about Valuism – my life philosophy. This post is the most technical of the sequence. Here are the first, second, third, and fifth parts of the sequence.

This is the fourth of five posts in my sequence of essays about my life philosophy, Valuism – here are the first, second, third, and fifth parts (though the last link won’t work until that essay is released).

I find robots to be a useful metaphor for thinking about human intrinsic values (i.e., things we value for their own sake, not merely as a means to other ends).

Imagine that you’re programming a very smart robot. One way to do this is to give the robot an “objective function” (or “utility function”). This is a mathematical function that takes as input any state of the world and outputs how “good” that state of the world is. Suppose that the robot is programmed so that its goal is to get the world into a state that is as good as possible according to this objective function.

Imagine that, in this particular case, the robot’s objective function is separable into different distinct parts: i.e., the robot cares about multiple sorts of things. We can think of these as the robot’s intrinsic values. For instance, maybe part of its objective function says that it is good to help others, another part of its objective function says it is bad to cause pain to others, and a third part says that it is bad to deceive others. Now we can describe the robot’s intrinsic values as being “help others,” “don’t cause pain,” “don’t deceive,” and so on.

It may be the case that the robot would form intermediate goals (such as “open that door”), but the goals would ultimately be oriented toward its intrinsic values (e.g., the goal of helping others).

A neural net could be used to control the robot’s behavior, learning (based on the consequences of each action it takes) to predict which actions will lead to a good world rather than a bad one (according to its objective function).

Unlike this robot, we as humans don’t have a utility function that describes what we care about. That means that with humans, things are way more complex. But if we consider the simpler case of a robot, it can help us observe a few interesting and important things about how our own (human) values work.

1. Knowing the origin of our intrinsic values doesn’t change them 

If this robot were smart enough to one day figure out that it has an objective function, even if it figured out part or all of what that objective function is, that doesn’t mean it would stop caring about what its objective function says is valuable. Even if it knew that a human programmed it to have that objective function, its values would stay the same: after all, the objective function describes (precisely and completely) what the robot cares about.

Similarly, if the robot discovered one day that its creator’s motives did not resemble the objective function the robot was programmed with, that doesn’t make the robot suddenly have the same objective function as its creator; it merely knows more about why it has the objective function that it does.

We humans are, to a shocking degree, in the same situation as this robot. Our intrinsic values are determined through some combination of genetics (honed by evolution), our upbringing, adult life experiences, culture, and our reflection. If we figure out what caused our intrinsic values to be what they are, that doesn’t stop us from valuing those things! We are, in a sense, a kind of mesa-optimizer. Evolution, which is itself an optimization process, created us. We ourselves are, to an extent, trying to optimize. What we were optimizing for is not totally unrelated to the optimization process of evolution (which selects for whatever helps genes propagate), but it is also not the same as that optimization process (otherwise, everyone would want to be constantly donating their sperm or eggs to sperm/egg banks).

Occasionally I encounter someone who thinks that because we were created via evolution, and evolution is a process that optimizes the number of descendants that a reproductively-successful species has, we as individuals should care about having lots of descendants. But this is a logical mistake: just because the process that created you was optimized to create X, that doesn’t mean that you yourself must value creating X. Just because evolution was optimized for spreading your genes doesn’t mean you should have that as your goal.

2. We can develop non-intrinsic values out of our intrinsic values

A robot can develop values that are not intrinsic values. For instance, maybe the robot learns a rule that it should avoid taking certain types of actions (because they, on average, lead to negative value according to its objective function). This rule works well, which is why it learns it. Making an analogy to the way humans work, we can now say that the robot “values” avoiding these behaviors, but avoiding them is not an “intrinsic” value – they are just a means to an end to attain its deeper values. Like humans, the robot could end up in a situation where its instrumental values and intrinsic values diverge.

The environment could abruptly change such that, in the long term, the robot would produce a higher value of its objective function if it no longer avoided those (previously punished) behaviors – but because it follows the rule of avoiding them, it never learns that. We see this sort of behavior in people in many ways. One example is that victims of abuse sometimes adopt self-protective behavior that helped them survive in those abusive relationships, but that cause problems in future relationships, making it harder to become close to the new (much kinder) people in their life.

3. Robots might maximize expected value, but humans don’t

When designing such a robot, a natural choice for its decision rule would be to have it try to maximize the expected value of its objective function. That is, for each action, it would effectively be evaluating, “On average, how good will the world be according to my objective function if I take this action compared to if I take the other actions available?”

Humans don’t do this; we deviate from what an expected-value-maximizing agent would do (as has been well documented in the behavioral economics literature and cognitive biases literature). Interestingly, the Von Neumann–Morgenstern utility theorem shows that any agent that makes choices so as to maximize the expected value of ANY utility function will satisfy four basic axioms in its behavior. Since real human behavior doesn’t satisfy these axioms, this provides evidence that our behavior is not well modeled by trying to maximize the expected value of some utility function.

So what do humans actually do, instead? It seems we have a variety of forces that influence our behavior, including:

• Basic urges (such as the urge to go to the bathroom)
• Built-in heuristics (such as conserving energy unless there is a reason not to)
• Habits (if every time recently when we’ve been in situation X we’ve done Y, we’ll likely do Y the next time we’re in situation X)
• Mimicry (if everyone else is doing something or expects us to do it, we’ll probably do it too)
• Intrinsic values (the things we fundamentally care about as ends in and of themselves)
• Plus others as well.

Our behavior arises from a variety of interlocking algorithms (running in our brains and bodies), and these algorithms aim at different things (conserving energy, gathering energy, and so on). A human is not a system with a unified objective.

This piece was drafted on February 5, 2023, and first appeared on this site on May 7, 2023.

You’ve just finished the fourth post in my sequence of essays on my life philosophy, Valuism. Here are the first, second, third, and fifth parts in the sequence.

People that this essay is not about

Some Effective Altruists believe that objective moral truth exists (i.e., they are “moral realists”). They think that statements like “it’s wrong to hurt innocent people for no reason” are the sort of statements that can be true or false, much like the statement “there is a table in my room” can be true or false.

I disagree that there is such a thing as objective moral truth, but I at least understand what these folks are doing – they believe there is an objective answer to the question of “what is good?” and then they are trying to figure out that answer and live by it. 

This usually ends up being some flavor of utilitarianism plus maybe some moral uncertainty giving some weight to other theories such as protecting rights. In the 2019 EA survey, 70% of EAs identified with utilitarianism (though this survey did not distinguish between those who do believe in objective moral truth and those who don’t believe in objective moral truth but have utilitarian ethics anyway). I think this group of EAs that believe in objective moral truth is mistaken but that they are being coherent. They are the first group listed in the poll I took below, and they are NOT the group I am focusing on in this post. 

The flaw I see:

The group I am focusing on is represented by the second bar in the poll above. Many (most?) Effective Altruists deny that there is objective moral truth or think that objective moral truth is unlikely. But then I still go on to hear quite a number of such EAs say things like:

• “We should maximize utility.”

• “The only thing I care about is increasing utility for conscious beings.”

• “The only thing that matters is the utility of conscious beings.”

• “The only value I endorse is maximizing utility.”

(Note that by “utility” here, they mean something like happiness minus suffering, not “utility” in the Economics sense of preference satisfaction [unless they are preference utilitarians] or the Von Neumann–Morgenstern theorem sense.)

I find these statements by Effective Altruists very strange. If I try to figure out what they are claiming, I see a few possible disambiguations:

Possibility 1 – Contradictory beliefs: they could believe that maximizing utility is objectively good even though they don’t believe in objective moral truth – which seems to me to be a blatant contradiction in their beliefs. Similarly, they could be claiming that while they have other intrinsic values, they think they SHOULD only value utility (and should value all units of utility equally). But then, what does the word “should” mean here? On what grounds “should” you if there is no objective moral truth?

Possibility 2 – Misperception of the self: they could be claiming that while there is no objective answer to what’s good, the only intrinsic value they have (i.e., the only thing they value as an end in and of itself, not as a means to an end, that matters to them even if it gets them nothing else) is the utility of conscious beings (and that all units of utility are equal). In other words, they are making an empirical claim about their mind (and what it assigns value to).

Here I think they are (in almost every case, and perhaps in every single case) empirically wrong about their own mind. This is just not how human minds work.

If we think of the neural network composing the human mind as having different operations it can do (e.g., prediction, imagination, etc.), one of those operations is assigning value to states of the world. When people do this and pay close attention, they will realize that they don’t value the utility of all conscious beings equally and that they value things other than utility. While I can’t prove there is literally no such person on earth that only has the intrinsic value of utility, even for the most utilitarian people I’ve ever met, when I question them, I discover they have values other than utility.

And it stands to reason that human minds (being created by evolution) are not the sort of things that are likely to only value the utility of all beings equally. For instance, just about everyone I’ve ever met would be willing to sacrifice at least 1.1 strangers to save one person they love (even if they think that person wouldn’t have a higher than average impact or a happier-than-average life). I certainly would, and I don’t feel bad about that!

One very strong intrinsic value I see in the effective altruism community is that of truth – many EAs think you should try never to lie and are suspicious even of marketing. They sometimes try to justify this on utilitarian grounds (indeed, it can often be beneficial from a utilitarian perspective, not to lie). But this sometimes seems like rationalization – a utilitarian agent would lie whenever it produces a higher expected value of utility (but potentially only if it was using naive Causal Decision Theory (CDT) – H/T to Linchuan Zang for pointing this out), whereas many EAs make a hard and fast rule against lying (saying you should try to NEVER lie). This is easily explained as EAs having an intrinsic value of truth that they don’t want to accept as an intrinsic value (and so try to explain in terms of the “socially acceptable” value of utility).

As a side note, I find it upsetting when EAs try to justify one of their (non-utility) intrinsic values in terms of global utility because they think they are only supposed to value utility. For instance, an EA once told me that the reason they have friends is that it helps them have a great impact on the world. I did not believe them (though I did not think they were intentionally lying). I interpreted their statement as a harmful form of self-delusion (trying to reframe their attempts to produce their intrinsic values so that they conform to what they feel their values are “supposed” to be).

Possibly 3 – Tyranny of the analytical mind: they could be saying that while they may have a bunch of intrinsic values, their analytical mind only “endorses” their utility value. But what does “endorse” mean here? Maybe they mean that, while they feel the pull of various intrinsic values, the logical part of their mind only feels the utility pull. But then why should their analytical mind have a veto over the other intrinsic values? Maybe they believe their other intrinsic values are “illogical,” whereas the utility value is logical. But on what grounds is that claim made? If they could prove logically that only utility mattered, wouldn’t we just be back to claim (1) that there is objective moral truth, and they don’t believe that? 

Intrinsic values are just not the sort of thing that can have logical proof, and if they are not that sort of thing, then why give preference to just that one part of your mind? I’m genuinely confused.

Possibility 4 – Maybe they mean something else that I just don’t see. What else could they mean? I’d love to know what you think (or if you’re one of these people)!

It’s certainly possible that there are very sensible interpretations for their claims that I’m just not seeing.

In conclusion, for Effective Altruists who think there is objective moral truth, I think they are wrong, but I understand what they are doing (this post is not about them). But for ones that don’t believe in objective moral truth (which I think is the majority?) I think they are making some kind of mistake when their sole focus is utility. Of course, I could be wrong.

My personal philosophy – which I call Valuism (and which I am working on an essay about), attempts to deal with this specific philosophical issue (in a limited context).

But in the meantime, I’d love to hear your thoughts on this topic! What do you think? If you are an EA who doesn’t believe in objective moral truth, but you’re convinced that only utility matters, what do YOU mean by that? And even if you don’t identify with that view, what do you think might be happening here that I might have missed or misunderstood?

Thanks for reading this and for any thoughts you are up for sharing with me!

Summarizing responses to this post

Edit (1 September 2022): after posting an earlier draft of this post on social media, there were hundreds of comments, some of which tried to explain why the commenter is utilitarian despite being an anti-realist, or presented alternative possibilities not delineated in the original post.

One thing that’s abundantly clear is that there is absolutely no consensus on how to handle the critique in the above post. There are a really wide variety of ways that people use to try to explain why they identify with utilitarianism despite not believing in objective moral truth.

Here are some of the most common types of responses given:

1. Responses related to Possibility  1 (i.e., addressing “contradictory beliefs“)

      1.1 Accepting contradiction: many people have contradictory beliefs (and contradictory beliefs may be no more common in moral anti-realist EAs than in other people), and some people are willing to lean into them. As one commenter put it: “many sets of intuitions are *wrong* if you take coherence as axiomatic.” Some people are just okay with self-contradiction.

      1.2 Beliefs that aren’t actually contradictory: my explanation of Possibility 1 might interpret  “we should maximize utility” differently than some people who say that phrase mean it. Here are potential some interpretations by which that statement might actually be consistent with anti-realist views:

           1.2.1 Personal preference: some people do not intend for statements like “we should maximize utility” to be representative of moral truth but instead mean it as an expression of a personal preference that they have for maximizing utility, or an expression of the fact that they will avoid feeling reflexively guilty if they aim to maximize utility, or a statement that they will have a positive emotional response if their focus on maximizing utility. However, these responses still seem to fall victim to another critique from the post, which is the arbitrariness of giving preference to certain feelings/preferences over other ones. 

           1.2.2 Metaethical constructivism: this is defined as “the view that insofar as there are normative truths, they are not fixed by normative facts that are independent of what rational agents would agree to under some specified conditions of choice” (source). Some say this is “best understood as a form of ‘expressivism’“. Constructivism seems compatible with both moral anti-realism and utilitarianism, but it’s unclear to me how many effective altruists would hold this view (I think very few). 

           1.2.3 Valuing a different kind of utility: some people may mean “we should maximize utility” in reference to a different kind of “utility” than the classic hedonistic utilitarian interpretation of the word. For example, “utility” is sometimes used to mean a “mathematical function serving as a representation of whatever one cares about.” By such an interpretati, if someone says they are trying to maximize utility they are presumably referring to maximizing their own utility function (rather than some objective one) – and so they are not the focus of this post.

2. Responses related to Possibility 2 (i.e., “misperception of the self“)

      2.1 Second-order desires: people might not be misperceiving themselves at all but might instead be talking about second-order desires or desires about desires. As one commenter put it: “It might be that, though someone empirically does NOT possess desires consistent with maximising the utility of conscious beings, they possess the desire to possess these desires. They want to be the sort of person who does have a genuine utilitarian psychology, even if they don’t possess one now. This may explain the motivation to act as a utilitarian (most of the time) [despite being a moral anti-realist].” Though in this case, it’s unclear why they would want to or think they should give those second-order desire preference over their first-order desires.

      2.2 Unshakable realist intuitions: people might be acting and/or feeling as if utilitarianism is true while also believing (upon reflection) that moral realism isn’t true. One person commented that “many of our intuition[s] are based on a realist world even when rationally we do not believe in one, so it is easy to accidentally make arguments that work only in a realist world, and then try to rationalize the argument afterwards to somehow work anyway.”

      2.3 Mislabeling one’s metaethics: instead of misperceiving what they value, some people might be mislabeling themselves as moral anti-realists even though they aren’t. In other words, some people who call themselves anti-realists might actually be moral realists without realizing it (e.g., because they haven’t reflected on it). One commenter thought that this would be a common phenomenon: “They are expressing a real, but subjective, truth ‘It is true to me that everyone should maximize utility’…I think that ‘deep down’ you will find that in fact most effective altruists and indeed most people are moral realists but under-theorized ones. Even the anti-realists tend to act as if they were moral realists.”

      2.4 Choosing one’s own values: some argue that you can choose your values for yourself (though it’s unclear by what process one would make such a choice, or whether such a choice really can be made – it may hinge on what is meant by “values”). As one of the commenters put it: “It seems like you are assuming in [Possibility 2] that there is an objective answer to what a mind values, e.g. based on how it behaves. For one thing, it’s not clear that that is right in general. But a particular alternative that interests me here: one could have a model where one can decide what to value, and to the extent that one’s behavior doesn’t match that, one’s behavior is in error.” In other words, according to this view, maybe an individual themselves is the only person who can define their intrinsic values, and there is no objectively correct opinion for them to hold about this. But then, by what criteria (or based on what values) is a person deciding on what their values should be?

3. Reasons why Possibility 3 (i.e., “tyranny of the analytical mind“) may not be a confused approach

      3.1 Identifying with the analytic part of the mind: some people feel that choosing to endorse a particular framework (and choosing to endorse some values over other ones) is part of who they are – part of (or even the most important part of) their self-concept. In other words, the reflective part of them making that choice feels to them like it is “who they are” more so than other parts of them that have other preferences. Here’s how one person explained it: “For my part, the part of my mind that examines my moral intuitions and decides whether I want to act on them feels about as ‘me’ as anything gets.” Another person thought that ​endorsing some values over others makes sense because many people think that their “best” self would live “in accordance with the judgments they make based on arguments and thought experiments.” Another proposed explanation for people being guided by the analytic mind is that being guided in this way might be a normal feature of human psychology (which at least one person saw as needing no further explanation). Yet another explanation put forward was that some people can have a completely arbitrary “personal taste” for giving their analytical mind a veto over other parts of their mind (and, according to this argument, those people don’t need a further justification beyond their arbitrary taste).

      3.2 Simplicity and coherence meta-values: having fewer intrinsic values or having fewer intrinsic values that one allows to dictate their behavior can (some argue) be justified by having an intrinsic value of coherence, simplicity, or consistency. As one commenter put it: “I genuinely think I just have utilitarian intrinsic values. [It seems] relevant here that I also value coherence (in a non-moral sense, probably as an epistemic virtue or something), so if I find myself thinking something that is incoherent with another value of mine, I can debate & discard the less important one.” 

Possibility 4: Moral uncertainty

      4.1 Meta-moral uncertainty – believing that realism might be true: people who don’t identify as moral realists might still feel there is some possibility that moral realism is correct and might act as if it was correct (at least to some degree – say, in proportion to how much weight they give this possibility compared to other action-guiding beliefs). As one commenter put it: “Why do I keep donating (and doing other EA things), albeit to a lesser extent [since switching from moral realism to moral anti-realism]? The main reason is (meta) moral uncertainty: I still feel that it is possible that moral realism is correct, and so I think it should have some say over my behavior.”

      4.2 Misinterpreting moral uncertainty as anti-realism: People who think that their own beliefs are not necessarily objectively true (due to moral uncertainty) might conclude that they must be moral anti-realists, but they might be mistaken in calling themselves that. As one commenter explained it: “believing in moral objectivity is different from believing we are actually able to parse the true moral weights in practice.”

Possibility 5: Precommitment and cooperation arguments

      5.1 Benefiting from pre-committing to impartiality: some argue that acting as if classical utilitarianism is true might be justified on grounds related to resolving collective action problems (without having to believe that moral realism is true). For instance, one commenter wrote: “Being impartial between oneself (and one’s friends / family) vs. random people isn’t something that any human naturally feels, but it’s a ‘cooperate’ move in a global coordination game. If we’d all be better off if we acted this way, then we want a situation where everyone makes a binding commitment to act impartially. It’s hard to do that, but we can approximate it through norms. So EAs might want to endorse this without feeling it.” Though presumably, if this was the justification for utilitarianism, they would then switch to a different moral theory if they thought it better solved collection action problems (e.g., if they came to believe virtue ethics better solved collective action problems).

      5.2 Benefiting from pre-committing to preference utilitarianism: some commenters pointed out that preference utilitarianism could also be justified on self-interested grounds (this post was not intended to be about other forms of utilitarianism such as preference utilitarianism, but it was edited to clarify that only after some people had started commenting). As one commenter put it: “If we’re viewing morality as playing a counterfactual game with others, we should take actions to benefit them in a way essentially identically to preference utilitarianism. That doesn’t require any objective morality, it only requires self-interest and buying into the idea that you should pre-commit to a theory of morality that, if many people embraced it, would increase your personal preferences.” Though in such cases (if they were actually optimizing for self-interest), it seems strange they would choose a moral theory where their interests count equally to people they will never encounter and never be in collective action problems with. (Some might argue that this would make more sense if the person endorsed a form of multiverse-wide cooperation via superrationality, though it’s unclear how this resolves more concrete/real-life collective action problems).

Possibility 6: Social forces – as Tyler Alterman put it (when I was discussing this post with him – he’s named here with permission): “[I felt] that [for some EAs] their actual beliefs were at odds with the cultural norms of other smart people (EAs) that they felt alignment with, so they stopped paying attention to their actual beliefs. I think this is what happened to me for a while. There was an element of wanting to fit in. But then there is an element of – there are so many smart people here [in EA]… EA is full of Oxford philosophers – they must have figured this out already; there must be some obvious answer for my confusion. So I just went along with the obligation and normative language and lifestyle it entailed.” Social forces can be powerful, and in some cases, an explanation for human behavior can be as simple as: the other people around me who I respect or want the approval of do this thing or seem convinced this thing is true, so I do this thing and am convinced it is true.

This essay was first written on August 14, 2022, first appeared on this site on August 19, 2022, and was edited (to incorporate a summary of people’s responses) on September 1, 2022, with help from Clare Harris.

The claim can be quite hard to argue against because it has a certain vagueness that makes it hard to pin down what (if anything non-trivial) is really being claimed. Regardless, here are eight arguments I put together against the idea that everything we do is “selfish” or just for our own pleasure:

1. There are many actions that we take automatically and thoughtlessly due to repetition – not because of wanting or pleasure. Consider habits like brushing our teeth or sticking our phone in (for example) our left pocket (as opposed to our right one) when we’re done using it . These sorts of behaviors can be so automatic that we forget immediately afterward whether or not we’ve done these things. The point is not that these habits aren’t useful, just that at some point, we come to do them automatically without even considering whether they benefit us or not (as you may dramatically learn as you drop your phone on the ground when wearing shorts without pockets, thoughtlessly executing the “phone in pocket” habit).

Some other examples of automatic behaviors that we usually do without having any apparent desire/wanting/pleasure/pain involved: breathing, continuing to walk (once we’ve begun to walk), balancing, swallowing (once food is chewed), looking towards a sound (when there is an unusual but non-threatening noise), and social mirroring of body language.

2. Evolution didn’t select for humans based on how much they did what they “wanted” or based on who maximized their own pleasure. Rather, it selected for those whose genes spread most. Selfishness and pleasure are important tools that evolution used, but not the ONLY motivator. For instance, genuine altruism and a sense of obligation towards kin and allies can provide substantial evolutionary advantages!

3. Which pleasure are we talking about? For instance, we clearly sometimes forgo more pleasure now in exchange for extra pleasure later (e.g., by getting work done early) and other times sacrifice long-term pleasure for the short term (e.g., playing video games instead of studying when you have a big test tomorrow). So if people are just maximizing for their own pleasure, which pleasure are they maximizing for?

You might be tempted to reply, “that’s because we’re just adding up the pleasure across time to decide what to do.” But it seems clear that some people aren’t doing this (e.g., drug addicts who know their life is being ruined but sacrifice everything for the next fix). Many experiments in behavioral science and behavioral economics also contradict the idea that people are merely happiness maximizers. It’s too simple to say we “just do what gives us pleasure.”

4. “Wanting” shouldn’t be conflated with something bringing pleasure or reducing pain. They are quite correlated (since we tend to want pleasurable things), but pleasure and wanting are distinct. There are things we can really want (e.g., to “one day understand a mysterious scientific principle,” or “to keep promises” or to have certain things happen after we die) which are not about our pleasure.

Some neuroscience papers claim that “wanting” and “liking” can even be separately stimulated in rat brains. Whether scientists know how to do this or not, it seems we sometimes want things because they bring pleasure, but other times we just WANT them, so wanting and pleasure are not identical.

5. If the claim is weakened to say that we humans always do things that we have SOME sort of motivation to do, then (interpreting “motivation” broadly) the claim is trivially true. But it also doesn’t say anything – it’s right just by definition. Motivation is not identical to pleasure or wanting. So defining “wanting” to do something as having ANY motivation to do a thing doesn’t work because it renders the argument trivial. Similarly, if “self-interest” or “wanting” is just defined to be any pattern of brain activity that causes us to act, or any form of motivation at all regardless of what sort it is, then it is true (by definition) but also adds no information. What’s the point of even making the claim if it’s true by definition? In those such cases the claim can be actively misleading because “self-interest” has connotations to most people (even if you try to define those connotations away).

6. People sometimes do things that they know will bring them more pain than pleasure. For instance, a protestor who uses gasoline to set himself on fire might feel a spark of pleasure just before he lights the match, but he knows he will tremendously suffer until death. Or consider someone who takes an action for a social cause even though they know it will likely lead to spending the rest of their life in prison. Clearly, the person is sacrificing more happiness than they are gaining by such an action, yet some people do act in this way.

7. If we imagine a person who is extremely altruistic because they love making others happy, and we claim they’re “selfish” because they are doing it just to feel good, this is a very non-standard way to use the word “selfish.” It insinuates their behavior is somehow less good and is misleading in conversation. Of course, we can define words however we want, but if we define them in a way that is different than how others use a word, it makes discussion difficult and confusing.

What work does the word “selfish” do to explain things here? It’s clearer to just say (in this case) “the person is motivated by their love of helping others” and leave it at that. Most people would call that “altruism” (not “selfishness”) upon knowing all the details.

8. When we try to make the claim precise, it’s hard to do so (and, unfortunately, few I’ve encountered making the claim bother to try). We’re clearly not always maximizing long-term pleasure, but nor are we always maximizing immediate pleasure. Claiming we “always do what we want” is not the same as claiming “we always do what is pleasurable.” So maybe we just try the former claim?

If we try to restrict the claim to not be about pleasure or pain and just say, “we do what we want,” then how do we explain our numerous subconscious behaviors? And how do we define “want”? People often say they didn’t do what they “wanted,” so we can’t use colloquial definitions.

If “want” is broadened too much, then we’re back to just claiming that we do what we’re motivated to do; that is, we’re making a trivial definitional claim. So what is really being claimed by “we only ever do what we want”?
I think either nothing interesting or something false.

Now, all of this being said, clearly, people often DO act based on however they “want” to act (by a reasonable definition of “want”). And very often, people do act in such a way as to seek pleasure and avoid pain. It’s just that not all human actions fit that description, which is what the “everything is selfish” crowd claims.

To finish up, I’ll attempt to take the other side/steel-man the claim that “everything we do is something we want.” I think there is a psychological state of “desire for things to be a certain way” that drives many (though not all) of our actions. This desire for things to be a certain way is not the same as pleasure (though is often related to it) and the way we want the world to be is not always the way we think will make us happiest (though it often is). So, although I think that the generalization made by psychological egoism is false, I do think it’s approximately true, in a certain sense, a decent amount of the time.

This essay was first written on November 9, 2021, and first appeared on this site on November 12, 2021.

Here are a number of reasons why altruism is often rational:

I. People can value altruism for its own sake:

1. Intrinsic values: as a psychological fact, most humans intrinsically value at least some things as ends (not merely as means to other ends) that are not about their own gain. For instance: people may value the reduction of suffering around the world or the flourishing of the people in their country. 

2. Warm glow: most humans find that it gives them happiness to do altruistic acts. I call this “the Lucky Fact” about human nature. It’s both important and very lucky (i.e., it didn’t necessarily have to be this way if our evolution had taken a different path). We feel good to see positive feelings in the people we like, and we feel good about ourselves when we cause good feelings.

II. Genuine altruism is also instrumentally useful:

3. Evolution: there are multiple reasons evolution programmed most of us with genuine altruism, even though it optimizes for gene spread.

Altruism is rewarded in settings of:

i) raising children

ii) iterated games

iii) tribal loyalty, with punishment of defectors

iv) deception detection

4. Relationships: altruistic people tend to have stronger, happier, more goal-aligned, and mutually beneficial relationships. Although, in theory, a purely selfish person can have highly beneficial relationships, it is much harder to make these expedient tit-for-tat relationships work.

5. Pre-commitment: suppose that there was a world of highly rational, purely selfish beings. If they were able, they might pre-commit (jointly, as a group) to become partially altruistic as a way to help solve difficult collective action problems. By uniting goals, they mutually gain.

This piece was first written on December 27, 2020, and was first released on this site on February 25, 2022.

(a) a set of rules (explicit or implicit) that are made up by humans,

(b) a scoring system (explicit or implicit) for determining how players are doing or for deciding who wins,

(c) participants who are trying to increase their “score,” and

(d) a game context (outside of which the game rules stop applying).

So, by this definition, games include chess, poker, football, and tennis, but also things like:

• money games (e.g., competing with friends and acquaintances to have a more expensive-looking car/watch/suit)

• altruism games (e.g., billionaires outbidding each other in charity auctions)

• coolness games (e.g., choosing clothing to demonstrate that your taste is trend-setting rather than trend-following)

• intelligence games (e.g., Oscar Wilde verbally jousting with his friends)

• sexual games (e.g., a man trying to seduce a woman while maintaining plausible deniability and her playing hard to get despite her intense attraction to him)

• strength games (e.g., boys wrestling after school)

• legal games (e.g., lawyers using every tool they know to beat each other in a case)

• academic games (e.g., young academics trying to outcompete each other in terms of who can get the most papers published in the top 10 journals)

• knowledge games (e.g., two people debating a factual topic in front of others at a party, each trying to show that the other person is wrong)

• political games (e.g., trying to form a strong coalition and to make the opposing coalition look corrupt or incompetent)

• career games (e.g., optimizing your behavior for getting promoted, rather than, say, for accomplishing the purpose of your work role)

Our brains have a tendency to temporarily treat games as reality (a suspension of disbelief).

This is not a bad thing – it’s part of what makes games fun and motivating, and it gets us to try hard at them. Those that can’t or won’t do this suspension of disbelief tend to be bad at games. There’s little joy or motivation in games if we’re just thinking, “I’m moving this wooden peg, so this number goes up.” We must (at least temporarily) believe that the number MATTERS.

Games can be fun, rewarding, and motivating. For some people, game playing is one of life’s great joys. And games make learning more fun (in fact, games are fundamental to how we humans learn). Children invent and play many kinds of games that help them figure out adult behaviors. And gamification can make difficult activities feel easier (e.g., you can turn a difficult task into a game to make it more pleasant).

But, on the flip side, games also can become a big problem when we forget for too long that we’re playing a game. Or if we permanently swap them for reality. Or if we come to think that winning the game is what fundamentally matters.

Consider the way that game playing distorts different activities:

• Science gets really screwed up when it is treated as a game where we compete to publish, rather than being treated as a way to figure out the truth about reality. This is part of why science has so many false positives.

• Altruism gets really screwed up when it is treated like a game to prove you’re a good person rather than as a way to help others. This is part of why so much altruism is not effective at improving the world.

• Governments get really screwed up when politics becomes a game (where most of what matters is beating the other side) rather than treating politics as a way to get helpful policies implemented.

• Medical schools get really screwed up if they become a game of who can memorize the most and function the best without sleep, rather than being a means to train effective doctors.

• The startup world gets really screwed up when it becomes a game of who can raise the most capital or do the coolest sounding thing, rather than having a focus on making products that solve actual problems.

• News gets really screwed up when it becomes a game about who can get the most clicks rather than as a means to spread true information.

• Law gets really screwed up when it becomes a game about what companies and people can technically get away with, rather than as a means of enforcing agreements and protecting people.

Games can be small or large, great or terrible. The key thing is to not get stuck inside a game without realizing it. Sadly, many people spend their whole life stuck in a game, confusing it for something more.

Sometimes we have no choice but to play a game that we don’t value. But recognizing games for what they are can help us leave them when they are poorly aligned with what we actually care about.

It’s great to play games sometimes and to suspend your disbelief to make them more fun and motivating. But don’t forget for too long that you are suspending it.

Games are not reality, though they might have real-world consequences. The in-game scoring system (whatever it is) does not reflect what you truly, intrinsically value. The rules of the game are made up by humans and are not the fundamental constraints on what behaviors you can and can’t take (though there might be consequences for breaking the game rules).

Play games cognizantly.

This essay was first written on November 23, 2020, and first appeared on this site on June 23, 2022.

Yet, the consequences of these simple rules are profound. Various facets of the world are hard to understand without taking evolution into account. Since our brains and bodies (as well as the natural world) were literally crafted by evolutionary forces, without that evolutionary perspective, you’re missing an important angle.

Here’s a list of human traits that seem hard to explain without considering an evolutionary element (along with possible evolutionary explanations for each):

Note: I’m not an expert in evolutionary theory, and none of these theories are provable beyond a shadow of a doubt. It’s important to remember that not all traits we have are evolutionary adaptive (for instance, they could just be vestigial, or side effects of other adaptive traits, or deleterious but not given time to be squashed by evolution, or random genetic drift, or better explained as cultural phenomena). Think of these as hypotheses for why these aspects of ourselves and the world are the way they are.

Possible Genetic Explanations for Things We Observe

• Why we have a sour taste capacity: probably to detect food spoilage (e.g., the bad chicken may smell and taste sour), as well as under ripeness (think of an unripened apple) and excessive levels of acid (think of old wine that has turned to vinegar). Small amounts of sour taste fine (or even good) to us, but more pungent tastes are usually bad.
• Why we have a salty taste capacity: probably because we die if we don’t replenish our salt since we need it for ion/water homeostasis.
• Why we have an umami taste capacity: probably to encourage us to eat peptides and proteins, which are used in many parts of our bodies (e.g., the composition of muscle). This is the most recently discovered taste capacity that seems to have gained substantial acceptance.
• Why we have a sweet taste capacity: probably to encourage us to eat sugars since they have calories that we use for energy to stay alive. It might also exist to encourage us to eat starches (which can taste sweet as they break down in our mouth), but some now argue that there might be a unique starch taste capacity.
• Why we have a bitter taste capacity: probably to detect poisons (e.g., many non-edible chemicals taste bitter, such as pesticides).
• Why we start to like bitter foods more once we’ve eaten them for a while: possibly because our brains initially assume they might be poison, but after eating them for a while, learn they are fine after we don’t get sick (hence why people can come to enjoy black coffee even though most younger people find it disgusting when they first try it).
• Why some plants have psychedelic effects: probably because they are “trying” to prevent animals from eating them (if it’s a plant that does not spread its seeds by being eaten) or because it’s “trying” to get eaten (if it spreads its seeds by being eaten). Different psychedelic effects from different plants on different animals may be more or less pleasant and may or may not be dangerous for survival.
• Why plants contain caffeine: possibly because it seems to be a natural pesticide for some insects, and possibly because it is believed to attract some types of bees.
• Why we have an “itch” sensory system: probably to detect insects crawling on and landing on us (which is really important when you live outdoors, though not so much when you live indoors).
• Why, when something lightly tickles your skin and produces an itchy feeling, the itch tends to grow worse until it is scratched: possibly because while you may be able to detect an insect landing on you, you probably can’t tell if it’s sitting still on you, so the itching sense is reminding you to scratch (i.e., the bug still might be sitting there even if you can’t feel it).
• Why some people throw up in cars: possibly because some poisons cause a mismatch between what you see and what you feel (and in those cases, it makes a lot of sense to throw up to get rid of the poison). In other words, a mismatch between visual stimuli and proprioceptive stimuli would be an indicator that you may have been poisoned and should throw up to expel it. Being in a car happens to produce a similar mismatch of sensory input.
• Why we get sick of foods if we eat too much of them: probably a form of helpful encouragement to eat a varied diet (to get more balanced nutrition) so that we aren’t as likely to become malnourished.
• Why we sometimes have sudden cravings for non-addictive foods: possibly because the body senses we lack a certain important nutrient.
• Why symmetry is typically considered attractive: probably because various problems related to development in the womb and genetics that can affect survival or reproduction can produce asymmetry (in other words, symmetry is a little bit of evidence that a variety of things that could have gone wrong during gestation, didn’t).
• Why people find clear skin beautiful: possibly because some harmful infectious diseases can cause skin issues (though, of course, most skin issues are harmless)
• Why humans usually find babies cute: probably because it causes us to take better care of our babies and children (and our relatives’ children), which causes our genes to spread more effectively down the generations (and on the flip side, it causes us to be better-taken care of by adults when they have genes for liking cuteness – it is beneficial for the genes of both the adult and baby). Of course, that’s not the only force causing adults to treat babies well, and people seem to differ substantially in how value they put on cuteness.
• Why humans find baby animals cute: probably in part because we’ve purposely bred them to look cute to us (especially dogs) but also in part because babies of other species have features that resemble babies from our own specifies (e.g., helplessness, big eyes relative to the head, big head relative to the body, larger forehead, flatter face, etc.). I assume that the people I know who find (non-bred-to-be-cute) baby animals cuter than baby humans are outliers.
• Why there are so many diseases that are only likely to kill us when we’re old: probably because once we’re past the age of reproductive fitness, there is much less evolutionary pressure pushing to keep us alive (though there still is some, because we still might be useful for helping copies of our genes survive in others, such as in our grandchildren who we can help)
• Why obesity is common now in our society but doesn’t seem to have been a long time ago: probably because our drive to eat food (and calorie absorption systems) were designed by evolution for an environment where exercise was a built-in part of daily life, food was much scarcer, and the taste of food was not nearly as optimized for addictiveness as it is today. Hence many factors push us to eat much more than would have been typical. It’s possible that food was “healthier” way back when on average too, but that’s really tough to say, in part because diets now and back then both varied tremendously depending on location, and in part because optimal human nutrition is still not very well understood.
• Why sudden loud noises cause most people anxiety: probably because such sounds were a pretty good indicator of danger 20,000 years ago (e.g., a tree falling, lighting, a loud animal, an angry person)
• Why males are larger than females on average across nearly all large human groups: possibly because males used to battle with each other a lot more than females did (and so there was a sort of arms race in male size), and also possibly because males hunted more often than females (the larger average size of males may be an adaptation, e.g., for spear throwing, where body weight is helpful). Studies apparently show that the size difference between males and females within a species is correlated with the amount that males battle each other over mates, social status, or resources, though I can’t vouch for the reliability of these studies.
• Why your fingers get wrinkly when soaked in water for a long time: possibly to help us grip wet objects better (since according to at least one study, it seems to improve the gripping ability of wet objects) though this certainly could be a coincidence and some researchers argue against it.
• Why faces play such a prominent role in attraction even though most common facial differences have little or no direct useful function: probably because faces encode some other non-face related information (e.g., related to hormone levels and age) and also because they directly code a little bit of information about how attractive (on average) other people may find your son or daughter if you mate with that person (e.g., if birds find large beaks attractive their offspring will survive more if they have large beaks, all else being equal, even if beaks do not help in any other way).
• Why people who are blind from birth make facial expressions for some emotions (especially anger, contempt, disgust, fear, happiness, and sadness) that non-blind people can recognize at least fairly accurately: probably because these six social emotions are generally useful for survival when living in a group across a wide range of environments (by communicating important information to others) and so have been hardcoded by evolution in a form that others can recognize. It’s also possible that the studies that claim this finding were poorly designed, though.
• Why most people seem to have a gut reaction that incest is morally wrong: possibly an inbuilt mechanism to protect people from breeding with close relatives, which would have a much-elevated chance of producing genetic disorders.
• Why people get goosebumps: probably a (mostly) vestigial reaction for adjusting our hair follicles, which was programmed a very long time ago when our bodies had a lot more of it. Hair standing on end makes animals warmer when they are cold and increases their apparent size when afraid, to intimidate other animals (hence why cold and fear can create goosebumps). It may still be a bit useful for warmth because we have some hair on our bodies.

In the attached image, there are two non-existent women (generated by a face generation AI set to generate “brown hair white adult female”). If these were real people, they would likely be treated differently throughout their lives due to very minor differences in facial structure and skin smoothness.

Based on their faces alone, there’s no way to know with non-negligible accuracy which of these people (if they existed) would be more hard-working, more moral, wiser, or otherwise in possession of personal traits that we actually might care about. So why are humans so obsessed with faces? It seems likely to be caused by a combination of two factors:

(1) Runaway Sexual Selection

If peacocks find large tail plumage sexually attractive, then even if those feathers are not useful for anything else, that still creates an evolutionary selection pressure where those with larger tail plumage are more likely to pass on their genes (due to improved chances of mating). Similarly, if certain humans are found to be more attractive based on their faces, that creates an evolutionary selection pressure in favor of mating with those people because then their children have a higher probability of finding mating success themselves (and hence passing on their genes). This phenomenon reinforces faces being attractive (because those attracted to “good-looking” faces mate with “good-looking” people more often, therefore their children are more good-looking and so have an easier time mating, plus have a preference for “good-looking” faces).

Today, this selection pressure is likely much weaker than it once was since most people now end up having children. For instance, now the vast majority of people in the US live to be at least 50, and only about 15% of women and 25% of men in the 40-50-year age bracket are childless. In contrast, tens of thousands of years ago, far fewer would make it to the point where they would have children.

(2) Health Correlations

In the environment we lived in tens of thousands of years ago, some aspects of a person’s face correlated with the likelihood of the survival of their genes, in particular ones related to disease (some diseases impact the face), genetic disorders (some of them cause facial changes), and development in the womb (where abnormal development can cause facial changes).

The correlation between health and facial features is likely to be lower now than it used to be back then. Today, a person’s facial features might still help to predict someone’s age, their most probable gender identity, and whether they have certain health conditions – but, of course, none of these give us any legitimate justification for treating some people better and others worse based just on their face.

It has been found that certain facial features do correlate with hormone levels (like testosterone). While testosterone levels may play a role in aggression (they may be part of the explanation for why men are violent so much more often than women), using these small correlations to make judgments about any one person is going to be both highly inaccurate and highly unjust. Some other personality traits may also be very weakly correlated with a person’s facial features, but talking to the person for 20 minutes will, of course, give you dramatically more information about what that person is like. Yet, we are prone to read so much into the way a person looks.

Note: there is an additional effect when it comes to faces, which is that we are sometimes taught by our culture to value certain facial attributes more than others. This can act above and beyond the previously mentioned two factors.

We humans act as though faces are incredibly important despite them being a substantially arbitrary mask our genes have programmed for us. And they often impact how we humans treat each other, despite this unequal treatment being both unjust and unjustified. If you ever notice yourself treating someone less well because of their face, take note and adjust your behavior.

I am not saying that people should, for example, date people they are not attracted to. Obviously, attraction is an important part of relationships for most people, and the face is one part of what determines attraction. (You may also care about your children one day having attractive faces, so they can more easily find life partners they like.) Rather, what I’m saying is that we should be very wary about making negative inferences about any individual person based on their face (which is something that, unfortunately, the human mind seems to do often). The face says too little about a person’s character to be useful for predicting at the level of any individual.

This essay was first written on July 2, 2020, and first appeared on this site on December 17, 2021.

We are surrounded by more superstimuli than most of us realize.

Examples of superstimuli:

• food: Cheetos / skittles / McDonalds

• goal achievement: video games

• visual arousal: porn

• pair bonding: romance novels

• affection: dogs

• cuteness: puppies & kittens

• stories: TV

• beauty: photoshopped models

• gossip: celebrity magazines

• social approval: Facebook

There is nothing wrong with superstimuli in moderation, but they tend to be addictive, and they can make it harder to enjoy the natural (non-super) versions of those things, which can harm our quality of life. Often times the superstimuli give just *part* of the experience we really crave (like eating junk food that is really tasty and calorie-dense, without really providing satiety or nutritional value).

So be wary if you spend more time on social media than talking to loved ones (or if you own more than 20 dogs).

This piece was first written on July 1, 2020, and first appeared on this site on July 22, 2022.