2. Suppose aliens show up on earth that are far smarter than the smartest among us at all cognitive tasks. They have specific goals that aren’t fully aligned with ours, are completely unconstrained by human morality, and don’t value our survival. What happens next?

3. Suppose someone builds a hacking A.I. that is trained on all the public information about computer hacking ever written, can think and type 1000x faster than a human, plans far ahead, and deposits a fully operational copy of itself onto every sufficiently powerful computer it hacks. Each copy then hacks further computer systems. What’s the world like a month later?

4. Suppose someone wants to have complete control over the world. Unfortunately, they’ve created one hundred million software agents that each think like Einstein + Bill Gates + Elon Musk + Warren Buffet. The agents attempt to do exactly what is commanded without hesitation or limits. Can anyone stop them?

5. Imagine a being that is godlike in its capabilities (relative to us). Suppose its only desire is to have the world be a certain way with maximal probability. It will stop at NOTHING to make the world this way, and it won’t tolerate even the SLIGHTEST chance of things being different than it desires. Will the resulting world include a human civilization?

6. Suppose you can think, process information and act 100,000 times faster than other humans. That means if you spend a day making and executing a plan, that’s equivalent to someone else spending 270 years on it. Your goal is to become world dictator. Can you do it?

7. Scientists discover how to create bug-sized self-replicating robots that out-compete natural life. These bug robots each try to maximize their own objective function. Unfortunately, these robots have leaked out of the lab and are now in 20 countries. Every day they double in number. Would we be able to eradicate these robots?

8. There’s a machine so powerful it achieves any goal you specify. You give the goal to the machine as written text. You can’t control HOW it achieves the goal; it ONLY cares about literally achieving it EXACTLY AS STATED in the most efficient way possible, and it can’t be stopped once started. The machine may do absolutely anything not explicitly forbidden in order to achieve the specified goal. Will it usually be a good (or horrible) outcome if you give the machine an ambitious goal like “prevent all war”?

9. Scientists invent a new idea – the Omnicide Synthesis Box. It could have many societal benefits, but, on average, scientists estimate making it will bring a 5% chance of human extinction (though some say more like a 90% chance). Those scientists who are less worried decide to build it. Should the least cautious be the ones to decide on behalf of humanity?

10. Picture a swarm of locusts, each individually possessing the intelligence and strategic prowess of a grandmaster chess player, while coordinating with each other in perfect unison. Their creators have given them the goal of controlling all available resources, indifferent to the collateral damage. Who ends up with most of the resources?

11. Imagine an AI-powered/nanotech super-factory that produces whatever it’s programmed to at enormous speed and scale (whether commanded to make diamonds, super viruses, microchips, or assassination drones). What could the owner of that super factory do to the world?

12. A medical firm gives a superintelligence the goal of designing a cure for all diseases. The superintelligence realizes it’s not smart enough to do so, so it plans to first acquire most of the computing power on earth (as it predicts it will need this to achieve the goal it was given), and then it creates a billion far smarter copies of itself to solve the task. What if one very misspecified goal is all we get with a superintelligence?

13. Five companies are developing a very powerful tech that would be incredibly useful if done right but very dangerous if developed without extreme caution. They each believe they can develop it safely but don’t trust the others to do so. They all cut corners racing to be the one to make it. Do good intentions lead to horrible consequences when doing something safely is much harder than merely doing it?

(Two of the above were written by ChatGPT – I edited those two quite a bit, though.)

If you read this line, please do us a favor and click here to answer one quick question.

1. Parkinson’s Law: Work expands so as to fill the time available for its completion.
2. Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.
3. Gates’ Law: Most people overestimate what they can do in one year and underestimate what they can do in ten years.
4. Goodhart’s Law: When a measure becomes a target, it ceases to be a good measure.
5. Hanlon’s Razor: Never attribute to malice that which is adequately explained by stupidity (or, don’t invoke conspiracy when ignorance and incompetence will suffice, as conspiracy implies intelligence).
6. Acton’s Dictum: Power tends to corrupt, and absolute power corrupts absolutely.
7. Amara’s Law: We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
8. Benford’s Law: In a diverse collection of unrelated statistics, a given statistic has roughly a 30% chance of starting with the digit 1.
9. Betteridge’s Law: Any headline which ends in a question mark can be answered by the word ‘no’.
10. Brooks’ Law: Adding manpower to a late software project makes it later.
11. Chesterton’s Fence: Reforms should not be made until the reasoning behind the existing state of affairs is understood.
12. Claasen’s Law: Usefulness = log(technology).
13. Clarke’s First Law: When a distinguished elderly scientist states that something is possible, they are almost certainly right, but when they state something is impossible, they are probably wrong.
14. Cromwell’s Rule: Nothing but logical impossibilities have a prior probability of 0 or 1.
15. Cunningham’s Law: The best way to get the right answer on the Internet is not to ask a question, it’s to post the wrong answer.
16. Doctorow’s Law: When someone puts a lock on a thing you own, against your wishes, and doesn’t give you the key, they’re not doing it for your benefit.
17. Dunbar’s Number: Most people can’t maintain stable social relationships with more than 150 people.
18. Eroom’s Law: Drug discovery is becoming slower and more expensive over time, despite improvements in technology.
19. Gell-Mann Amnesia Effect: You’ll believe articles outside your area of expertise, even after acknowledging that neighboring articles in your area of expertise are completely wrong.
20. Gibson’s Law (or the Expert Witness Law): For each PhD (to use as an expert witness for one side) there’s an equal and opposite PhD.
21. Godwin’s Law: As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches one.
22. Morley-Souter’s Law (Rule 34): There is porn of it (no exceptions).
23. Greenspun’s Tenth Rule: Any sufficiently complicated C program contains an ad hoc, informally specified, bug-ridden, slow implementation of half of Common Lisp.
24. Hebb’s Law: Neurons that fire together wire together.
25. Hubble’s Law: Galaxies recede from an observer at a rate proportional to their distance to that observer.
26. Hume’s Guillotine (Is-Ought Problem): Normative statements (about what’s moral/immoral/right/wrong) cannot be deduced exclusively from descriptive statements.
27. Humphrey’s Law: Conscious attention to a task normally performed automatically can impair its performance.
28. Kranzberg’s Law: Technology is neither good nor bad; nor is it neutral.
29. Lamarck’s Principle (or “Use it or Lose it”): Use it or lose it (evolutionarily speaking, but also in the brain).
30. Lewis’s Law: The comments you’ll inevitably find on any article about feminism justify feminism.
31. Littlewood’s Law: Individuals can expect miracles to happen to them, at the rate of about one per month.
32. Maes–Garreau Law: Favorable predictions about future technology will fall at the latest possible date they can come true and still remain in the lifetime of the predictor.
33. Metcalfe’s Law: The value of a system grows as approximately the square of the number of users of the system.
34. Miller’s Law: To understand what another person is saying, you must assume that it is true and try to imagine what it could be true of.
35. Moore’s Law: Computation per dollar grows exponentially (or: number of transistors per circuit doubles roughly every 24 months).
36. Murphy’s Law: Anything that can go wrong will go wrong.
37. Alder’s Law: What cannot be settled by experiment is not worth debating.
38. O’Sullivan’s Law: All organizations that are not actually right-wing will over time become left-wing.
39. Pareto’s Principle (80/20 Rule): For many phenomena 80% of consequences stem from 20% of the causes.
40. Peter’s Principle: In a hierarchy, every employee tends to rise to his level of incompetence.
41. Poisson’s Law (or Law of Large Numbers): For independent random variables with a common distribution, the average tends to the mean as sample size increases.
42. Pournelle’s Iron Law of Bureaucracy: In bureaucracy, those devoted to the bureaucracy get control, those devoted to what it’s supposed to achieve lose influence.
43. Putt’s Law: Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand.
44. Rosenthal Effect (Pygmalion Effect): High expectations lead to an increase in performance, low expectations to a decrease in performance.
45. Schneier’s Law: Any person can invent a security system so clever that she or he can’t think of how to break it.
46. Shermer’s Law: Any sufficiently advanced extraterrestrial intelligence is indistinguishable from God.
47. Zipf’s Law: The frequency of use of the nth-most-frequently-used word in any natural language is approximately inversely proportional to n (few words are used often, most are used rarely).
48. Wirth’s Law: Software gets slower more quickly than hardware gets faster.
49. Sturgeon’s Law: Ninety percent of everything is crud.
50. Stigler’s Law: No discovery is named after its original discoverer, including this one.

This piece was first written on July 6, 2020, and first appeared on my website on May 30, 2026.

1. Looping the Question

“What don’t I know that I must know?” This Meta Question is your obsession. The answer to that Meta Question is itself a question, and it leads you to create your first data loop (say, conducting a series of interviews with experts). After sucking down much of what it has to teach you, you’re hearing the same answers again and again. Data loops are ephemeral, but this one has served its purpose.

“What don’t I know that I must know?” you ask again. The answer yields your next question, which leads to your next data loop (say, reading many articles). Then that stops being as useful, and you construct your next loop (e.g., conducting a survey) and another (e.g., watching people use your product) and another (e.g., getting criticism from UI experts) and another (e.g., studying the statistics of user behavior) and yet another (e.g., attempting to sell your product at different prices). For each question, you look to the many tools on your tool belt (e.g., user interviews, expert feedback, surveys, A/B tests, analysis of user behavior, etc.) to pick the tool that is most suited for answering it. You are a master asker of questions, an expert looper of the answers.

2. Programming the Machine

You are constructing a machine out of gears. Some parts are made of metal or plastic, but mostly they are made of bits or people. Every month the machine produces a series of outputs, some intermediate, some final. You inspect these carefully.

Do the outputs have the right weight, color, clarity, cost efficiency, quality, quantity? Which part of the machine is bottlenecking the rest? Which part could use lubrication, or gas, or computation, or money, or inspiration? Which part of the machine is grinding to a halt, working against the rest, rotting inside, or about to fly apart? You perform intuitive calculus. What gear has the highest derivative of output with respect to dollars (or hours) of further investment?

As you feed it, the machine keeps growing bigger and more complex. You no longer have a complete understanding of how it operates – nobody does. It has ever more parts that can break, greater momentum which resists changing direction, more fuel needed to move it, more of a mind of its own. You battle the exploding complexity while the metal/plastic/flesh/binary monster of your creation lurches forward, smoke and steam spewing from its exhaust pipes. If you expertly wield the tools at your disposal, you can still usually manage to nudge it a tad to the left or a bit to the right, making sure it stays roughly on course.

3. Polishing the Stone

At first, you have a crude stone with countless rough edges. You show it to everyone. It looks like beach trash. It does almost nothing. People are confused why you are so excited about it. No one can see the form you see, trapped inside (though your friends try to, or at least try hard to pretend that they do). “Who would want that?” people silently wonder as they nod at what you’re saying. And these people are correct – no one wants it. You’re peddling rubbish.

But you believe in your stone. With enormous effort, you smooth the stone’s thousand rough edges. “You didn’t really need to smooth edge #732”, someone points out. “One little edge makes almost no difference.” This person is correct. Smoothing any single point makes *almost* no difference. Yet by smoothing one thousand rough edges, you’ve created a beautiful gem. It’s now the best of its kind in all the world. It’s now desired. “The idea was obvious,” people mutter, “I could have thought of that.”

4. Outlasting the Jungle

You’ve just parachuted into a dense jungle. Everyone knows there are treasures of incredible value buried there – in many different locations, in fact – but nobody knows where the treasure is. To make matters worse, the jungle is crawling with snakes, spiders, alligators, hungry bears, angry bears, crazy bears, bats, spike traps, people-eating plants, dragons, and buffalo, plus far too many other treasure hunters like you.

The vast majority of those who enter the jungle will end up fleeing it empty-handed. After a few snake/bear/buffalo/dragon bites and a few miles (or hundred miles) of strenuously hacking through the jungle, without ever quite knowing which direction to go, most have had enough.

Occasionally, though, people leave the jungle with some treasure in tow. And every once in a while, a group emerges with what is truly a king’s fortune. There is no shortage of treasure hunters. But you are different than the others. You are prepared, courageous, and confident – and you learn quickly. You have plenty of funding, a knife gripped between your teeth, a crossbow on your back, a net slung across one shoulder, bear spray for the bears, dragon spray for the dragons, grenades in case things get really tough, a tiny army at your back, 22 different maps written by 22 former adventurers (that semi-contradict each other), and a determination of tempered steel.

You know that if you can stay in the jungle longer than all the others – if, when others are falling down and coming apart, you can rally and try ten more paths to glory – while gaining skills from each failed outing – you will vastly increase your chances of success.

But then again, as you look around, you see dense thickets full of thorns in all directions. The tree cover blots out most of the sun, casting long, foreboding shadows. There is a faint bellowing roar in the distance, but the pitch is too low to identify its direction. And sweat beads on your forehead. “Which way?” your tiny army asks you.

So you’re thinking of starting a startup? Become a looper of questions to know how to program the machine that will polish the stone while you outlast the jungle.

This essay was first written on December 22, 2018, and first appeared on this site on January 28, 2022.