“I’ve brought you here to play a game,” Omega says. “The well-being of humanity depends on your choices, so pay very close attention. You start with 100 credits.”

A screen suddenly appears with the number 100 on it. “To play the game,” Omega continues, “just think of some number of credits that you’d like to bet during the next round, and then push the blue button. You can’t bet more credits than you have remaining. Whatever number of credits you are thinking of when you push the button will be your bet for that round.”

A blue button appears.

“With each push of the blue button, there is a 99% chance that you win, in which case the credits you bet that round will increase by 10%. But there is a 1% chance that you’ll lose, in which case you forfeit all the credits you bet.”

“Let me give you an example,” Omega continues. “Suppose that you have 100 credits and bet 10 of them. If you win, which, remember, has a 99% chance of happening, your 10 credits will grow to 11 credits, so you’ll have 101 credits after that round. But if you lose, which, remember, has a 1% chance of happening, you will lose those 10 credits you bet, and so will end the round with only 90 credits. Is that clear?”

The (“maximizing”, “act”) utilitarian nods.

“Good. It’s also important to know that winning or losing in any particular round won’t affect the chance of winning or losing in any other rounds. And you can play as many rounds of the game as you like. Reality is paused while you are in this room, so if you want, you can play for a hundred years or a million years or however many rounds you choose. When you decide to stop playing, just leave the room through that door to return to your normal life. The moment you leave, I will adjust the world by adding an amount of positive utility proportional to the number of credits you have remaining. The amount of utility I will add to the world for each credit you end up with is equivalent to the utility of 1000 of the happiest lives humans have ever lived. That means there is a lot of utility on the line here, so don’t F this up.”

Omega disappears. The utilitarian thinks about the game for a while and performs some calculations.

“If I bet C credits on my first play, then the average number of credits I will get back from my first play is:

Average I get back after one play = 0.99 [chance of winning] * 1.10 [reward if I win] * C [amount I bet]+ 0.01 [chance of losing] * 0 [penalty if I lose] * C [ amount I bet] = 1.089 C ≈ C + 0.09 C

So that means that each time I play, the number of credits I get back is whatever I bet, increased by about 9% on average. So I maximize the average number of credits I end up with by betting all of them in my first play! If I bet 100 credits, then on average I’ll end up with 108.9, whereas if I bet 99 credits, then on average I’ll only end up with 108.811, so it maximizes expected value to bet them all.

However, nothing changes from one round to the next, since each round is statistically independent from the last. That means that every time I play, I should bet ALL my credits. And if I do that, my credits will grow exponentially, by about 9% with each play. That’s amazing!

Wait a moment…that doesn’t feel right. Isn’t it really risky to bet all my credits each round? I risk losing all of them. But that strategy does seem to maximize the expected total utility, so it is the most ethical thing to do.

Let me double-check my reasoning. Suppose that I bet a fraction F of my credits in one round, and I start that round with C credits. That means at the end of that round, on average, I have:

average after one round = 0.99 * 1.10 * F * C + 0.01 * 0 * F * C + (1-F) * C
= (0.99 * 1.10 * F + (1-F) ) C = C + 0.089 F C

This is clearly maximized when F is as large as possible, which is F=1, so I clearly should bet all my credits on the first round to maximize the average utility! That again seems to imply I should always bet all my credits on every round, since each round is identical and independent from the others, so the decision should always be the same as in the first round.

Hmm, still seems odd. To be sure, maybe I need to think about what happens if I play k rounds in a row, rather than just one.

Let’s suppose I play for two rounds in a row, and bet a fraction F1 on the first round, and a fraction F2 on the second round. Let M1 be the multiplier I happen to get on the credits the first round (1.10 with 0.99 probability or 0 with 0.01 probability), and M2 the multiplier on the second round (which is identical to but independent from M1). Then, after two rounds where I bet first F1 then F2, my average payout is:

average after two rounds = average[ M2 * F2 * (M1 * F1 * C + (1-F1) * C) + (1-F2) * (M1 * F1 * C + (1-F1) * C)] = ( 1 + 0.089 F1 + 0.089 F2 + 0.089^2 F1 F2) C

Which, since all the coefficients are positive, will always increase as F1 increases and as F2 increases. So to maximize the expected utility, we want to use the maximum values for F1 and F2, which are just F1=1 and F2=1, implying again that we should bet all our credits on each round even if we’re playing for two rounds!

If I bet all the credits I have at each round, then the expected value after playing k rounds will be my original starting credits multiplied by the product of the expected values of M1, M2, etc., since each is independent from the others (and the expected value of a product of independent variables is the product of the expected values). Therefore:

average after k plays = 100 * 1.089^k

Wow, so the expected utility really will grow exponentially, and the more times I play, the greater that utility will be, on average! And even if there is some other way to produce an equal amount of utility in this game, through some other betting strategy, this approach will clearly do it in fewer plays than any other – and since I myself get slight disutility being stuck here, forced to play this game, all utility (including my own as a tiny part of that) is maximized playing in the more efficient way.”

He triple and then quadruple checks his math. Satisfied that he’s really found the utilitarian solution, he starts to play the game, betting all of his credits in each round.

After 50 rounds, he’s up to 11,739 credits. After 100 rounds, he’s at 1,378,061 credits, equivalent to more than a billion happy lives. He’s extremely tempted to leave at that point with his utility winnings, but he rechecks his calculation again, and it tells him that the average utility maximizing action (i.e., the most ethical one according to his moral philosophy) is to keep playing. At round 118, he loses, and his credits are all wiped out. With no more credits to bet, he has no choice but to leave. He returns through the door to his normal life, and the world is no better off than when the game began.

His strategy of maximizing total average utility appears to be required of him as the most ethical action according to his utilitarian ethical beliefs, yet it seems to be the worst strategy possible in Omega’s game, guaranteeing that the lowest possible amount of utility will be created with 100% certainty.

Interestingly, I think the same scenario still works (i.e., presents a problem and seeming paradox for the utilitarian) even if Omega says that the game also has a chance of ending on its own (if you decide never to end it on purpose), with the probability of the game ending after n rounds are completed of 1/(n*(n+1)). So if you never end the game on purpose, the game has 1/2 probability of automatically ending after the first round, a 1/6 probability of automatically ending after 2 rounds, a 1/12 probability of automatically ending after 3 rounds, and so on. Other than that change, though, the rules are exactly the same (so you can still choose to end the game any time you want, it’s just that now the game might also end after one of the rounds automatically, even if you want to keep playing, and the game ending automatically has the same result as if you ended it on purpose). Note that this specific probability distribution (of the game ending automatically) is special because it implies that the expected value for the number of rounds played is still infinite, even though the game might automatically end after any round.

This piece was first written on April 7, 2018, and first appeared on my website on September 11, 2025.

(1) Human groups have pretty much always done unspeakably horrible things to other groups (e.g., think mass murder and numerous forms of oppression). And fairly often, while it is happening, this evil seems to be condoned, supported, or only passively opposed by a large and fairly culturally representative group, rather than by just a few very bad people.

Are we humans evil?

(2) On the other hand, the vast majority of people feel sympathetic when they see people suffering (even realistic depictions of suffering in movies), and are willing to help people they don’t know at some inconvenience to themselves in the right circumstances. For instance, if an elderly stranger passes out in an office building elevator when only one other person is there, most likely that other person would try to help, even if it caused them to be late to work. In other words, the vast majority of people are at least moderately pro-social, have some compassion for strangers, and will at least sometimes inconvenience themselves to help others when they don’t expect to gain anything out of it.

Are we humans good?

Originally I attempted to explain these phenomena using the idea of “mimicry” that people just copy what others do, so if people are harming others, people will copy and do it too, but I no longer think that really gets at the core of the issue (though I do think mimicry plays a certain role, as you’ll see).

Below is my attempt to reach a simplified explanation for the seeming contradiction between (1) and (2). This theory is by no means novel. If there is anything novel about it, it’s in the particular way of combining the various pieces, or trying to fill out certain details.

So what’s going on with our species?

We humans seem to automatically divide others into “moral groups.” In our minds, each of these moral groups has an implicit “moral weight.” A group’s moral weight determines:

• whether you think the fates of members of that group are important (high moral weight) or irrelevant (low moral weight)
• whether you see it as good to help a member of that group (high moral weight) or acceptable to harm a member of that group (low moral weight)

If harming a certain group (call it “group X”) is considered both acceptable AND happens to be useful, convenient, or a way to reduce perceived danger, then murder or subjugation of group X may ensue with little guilt. In other words, if one group assigns group X a low moral weight, and there is some additional reason why harming group X seems useful, then that harm may well occur.

The simplest example of this is the way we kill insects regularly without a second thought. But the distinction of being assigned very low moral weight is not just reserved for insects. We humans unfortunately sometimes do it to other human groups.

Some small percentage of people in our society view killing insects as morally unacceptable. Similarly, in societies where the majority generally assigns essentially no moral weight to group X and plans to massacre or oppress group X, there will usually be a minority of conscientious objectors who protest it. According to this model, this occurs because (for various reasons) this minority assigns a different moral weight to group X than the majority.

Here are some examples of common groups people might assign different moral weights to:

• “family member”
• “friend”
• “child (my own)”
• “child (not my own)”
• “person I am interacting with right now”
• “neighbor”
• “citizen of my country”
• “person I’ve met before but don’t really know”
• “person of my religion”
• “criminal”
• “white person”
• “black person”
• “murderer”
• “celebrity”
• “French person”
• “friend of a friend”
• “co-worker”
• “foreigner from somewhere unfamiliar”
• “human being”
• “cow”

People can fall into different groups, each group having varying moral weights, making it more complex than an in-group/out-group binary. If someone falls into multiple groups, then the moral weight assigned to that person is some complex combination of the moral weights of the different groups they belong to. In some cases, it seems to be the maximal moral weight of any group they belong to (e.g., a parent would usually be treated as being in the parent group regardless of what other groups they fall into), but sometimes we focus on the lowest moral weights (e.g., a work colleague who was also a murderer may end up getting a moral weight closer to that of a murderer than of any other work colleague).

I’ve identified six factors that seem to be predictive of how much moral weight a person assigns to a given group. Moral weight seems to be higher for:

1. Relation: groups that share a person’s genetics (e.g., “parent”). This has an obvious evolutionary explanation as a gene propagation mechanism. Many people feel we have the strongest moral obligations to our family members.

2. Enjoyment: groups that we enjoy interacting with (e.g., “friends” or “puppies”). For instance, people are more comfortable screwing over a person they dislike than a person they like.

3. Familiarity: groups we are highly familiar with or spend a lot of time with (e.g., “work colleagues”). Almost everyone would save a long time acquaintance over a stranger they’ve just met if forced to make a choice.

4. Similarity: groups we perceive as similar to ourselves or that we group ourselves with (e.g. “people of our own country” or “people of our own religion”). People feel a heightened duty to protect others from their country, religion, etc.

5. Safety: groups we are not afraid of or view as non-threatening (e.g., this would exclude “sharks” and “violent criminals”). People often feel justified in harming someone in “self-defense.”

6. Innocence: groups we feel are innocent rather than capable of being sinful or responsible for their actions (e.g., “children”). There is something that people find particularly repugnant about harming children, or anyone helpless.

As mentioned before, a classic example of a group who the majority view as having near-zero moral weight is “insect.” Insects (1) hardly share any genetics in common with us, (2) we don’t like interacting with them, (3) we don’t generally spend much time with them (at least, not on purpose), (4) we don’t perceive them as similar to ourselves, and (5) we often view them as a threat. The only thing they have going for them is that we may or may not view them as innocent, but I suspect a lot of people view them as guilty because sometimes they seem to be intentionally bothering us.

Chickens do slightly better than insects on these criteria. Those who interact with them tend to enjoy it a lot more than interacting with insects, and they seem more similar to us than insects, but mostly we treat them as having low moral weight as well (which helps explain why it’s so common to eat them).

To give a few more examples:

• Cats (in the U.S., anyway) do much better on the criteria 2-6 than chickens, so we assign them much higher moral weight than chickens.
• People of our own country do much better on criteria 2-4 than people of other countries.
• Family members usually do much better on criteria 1-5 than people of our own country.
• Violent criminals typically don’t do well on criteria 1-6, and, empirically, people often seem willing to strip rights away from this group, and in some cases, have them executed.

Circling back to our original question with this model in mind, how do we rectify the seeming contradiction that:

(1) most people are pro-social

(2) human groups do unspeakably horrible things to other groups

According to this model, the answer is simple. When one group assigns almost no moral weight to another group, they are willing to do horrible things to do that group. To them, it is simply not morally problematic, because that’s what moral weight MEANS. However, when that group has a reasonable amount of moral weight assigned (e.g. “person who works in my office building”), they are happy to be helpful and generally act pro-socially. A small percentage of people are truly sociopathy, and assign low moral weight to everyone. But the far more common thing is to assign high moral weight to some groups, and low moral weight to others (influenced by factors 1-6 above).

So, instead of: “are we humans fundamentally good, or not?”, the question becomes: “who are we good to?”

Now, this doesn’t mean that a group WANTS to destroy all groups they assign a low moral weight to. You may assign low moral weight to insects, but that doesn’t mean you spend your weekends trying to find insects to harm. All that low moral weight means is that you are WILLING to do harm if it happens to be advantageous. If an incentive to destroy or control the group is added (e.g., the group is viewed as threatening or useful), then the mass murder or mass oppression may begin.

If this simple theory is reasonably accurate, then it suggests that most of the greatest evils of humanity are not the evils of individual bad actors, but the evil of assigning little moral weight to other groups, combined with the view that it would be “useful” to harm that low moral weight group.

When these groups are across borders, we get war and mass murder. When these groups are ethnic, it can lead to genocide. When these groups are across gender, it can lead to the mass oppression of women or the murder of female babies. When these groups are across species, we get large scale mistreatment of animals in factory farms.

But where do these assignments of moral weight come from? Here are some possible factors:

(i) Mimicry: to a significant extent, we copy the moral weights that we are implicitly taught by the people around us, especially when we’re growing up. So if, as a child, we see that our family members don’t assign any moral weight to group X, we learn not to assign a moral weight to group X as well. This is, I think, where mimicry does come into play.

(ii) Bad actors: sometimes, individual bad actors that are influential in a group, call it group “A,” convince others in group A to assign less moral weight to another group X. This may be for their own selfish reasons, or because they themselves assign a low moral weight to X and want others to as well. The bad actor may cause group A to assign a less moral weight to X by appealing to factor (D), the idea that group X is very different than group A, such as by highlighting differences between A and X or causing group X to stand out even more. Another option is for the bad actor to try to leverage (E), the idea that group X is harmful or dangerous. They may also do this by taking advantage of factor (F), the idea that group X is not innocent, for example, by blaming something that happened a long time ago on that group, even if it poses no threat today.

(iii) Danger: stability and safety in our society seem to influence the assignment of moral weights. It seems that, on average, the less we can trust those around us, and the more we have to fear danger, the more narrowly we define which groups have a high moral weight, and the more we rely on factors A-F to decide who matters. For instance, if you are trying to survive during wartime and barely have enough to eat, you may be more likely to only assign a substantial moral weight to your closest family members, whereas as danger lessens, you may start increasing the moral weight that you assign to neighbors.

(iv) Genetics: It’s also possible that genetic differences exist as well in how we assign moral weights to others, since there is at least some genetic variation in quite a number of traits. For instance, it could be that some people are just born more compassion than others, and are therefore more prone to assign higher moral weights to more groups than others are, all else being equal.

There have been a number of attempts to cause people to more broadly assign higher moral weight, such as in certain Buddhist traditions, the Effective Altruism movement, and, perhaps, the Hippie movement. One common theme of such movements is an unusually large amount of vegetarianism and veganism since, as the expanding “moral circle” (as its sometimes called) begs the question of whether other species should be included.

Creating widespread change in the assignment of moral weight seems as valuable as it is challenging, it strikes me as an extremely valuable project. If the simplified model suggested above is at least modestly accurate, then, causing a more broad and uniform assignment of moral weight could go a long way toward preventing future evils that we humans tend to inflict on each other. What’s more, when we assign reasonably high moral weight to a group we do more than prevent harm: we tend to actively help it, even where personal benefit is unlikely.