Ten years ago Eliezer Yudkowsky wrote a post about cryonics where he was baffled by the fact that most young cryonicists heard about the concept and then decided to sign up. There was no extended questioning, no sales pitch, young (future) cryonics patients were simply exposed to the concept of cryonics by seeing it mentioned on television or the radio. He referred to this simple absorption of the idea as ‘clicking’, and desperately wanted to know what went into the seemingly magical click. EY hypothesized that rather than having an extra sanity gear in their head, it seemed more likely cryonicists had some insanity gear(s) missing.

Eliezer, you of all people should know there is no such thing as magic. You even say so in this exact post. User ‘bshock’ replies with his experience working a job where he signed people up for cryonics, writing of the reasons people reject it:

The first and largest by far tended to be religious, which is to say, afterlife mythology. If you thought you were going to Heaven, Kolob, another plane of existence, or another body, you wouldn’t bother investing the money or emotional effort in cryonics.

Only then came the intellectual barriers, but the boundary could be extremely vague. I think that the vast majority of people didnt have any trouble grasping the basic scientific arguments for cryonics; the actual logic filter always seemed relatively thin to me. Instead, people used their intellect to rationalize against cryonics, either motivated by existing beliefs (from one end) or by resulting anxieties (from the other).

Anxieties relating to cryonics tended to revolve around social situation and/or death. Some people identified so deeply with their current social situation, the idea of losing that situation (family, friends, standing, culture, etc.) was unthinkable. Others were afflicted by a sort of hypothetical survivor guilt; why did they deserve to live, when so many of their loved ones had died? Perhaps the majority were simply repulsed by any thought of death itself; most of them spent their lives trying not to think about the fact that we would die, and found it extremely depressing or disorienting when forced to confront that fact.

(Bolding mine)

I find this answer astonishing in its clarity, and frustrating in its prescience. Looking back on it after nearly two years of research it’s annoying to think if I’d been paying more attention I could have caught on to the importance of the fear of death earlier. It’s not that I didn’t think the fear of death was important, the problem is that I hadn’t understood how important it is to peoples ability to implement rationality. A great deal of what goes into the click is having a worldview that can soberly consider mortality. I actually hadn’t looked at that post again until sitting down to write this one. It’s encouraging to see the most credible answer points towards the thesis of this post: that the fear of death acts as a sort of master key for introductory rationality concepts. Examining the fear of death ties all the rationality basics together into a coherent framework, including:

• Map/Territory Errors
• Something To Protect
• Keeping Your Identity Small
• Atheism
• X-Risk

Why The Fear Of Death?

In ‘soft’ disciplines like psychology, it’s easy to confuse ourselves with compelling nonsense. Hypothesis space is vast, and we often pick from it by exploring some territory, seizing on a plausible idea, and then using a mix of confirmation bias and correlation to ‘prove’ our idea correct. Most of these proofs are worthless, you could construct another one of about the same justification to support a completely separate or even contradictory idea. So when we consider this topic it’s not enough to craft a plausible narrative and give it a body of connective conceptual flesh. We have to narrow the hypothesis space to try and give ourselves a better chance of landing in the right territory. In the service of that lets consider some other narratives which I often see cited to explain people bouncing off rationality and see if the fear of death stands out against them:

The Utility Narrative

Scott writes in his classic Extreme Rationality: It’s Not That Great that the basic reason why people aren’t interested in rationality is that it’s not useful:

Looking over history, I do not find any tendency for successful people to have made a formal study of x-rationality. This isn’t entirely fair, because the discipline has expanded vastly over the past fifty years, but the basics—syllogisms, fallacies, and the like—have been around much longer. The few groups who made a concerted effort to study x-rationality didn’t shoot off an unusual number of geniuses—the Korzybskians are a good example. In fact as far as I know the only follower of Korzybski to turn his ideas into a vast personal empire of fame and fortune was (ironically!) L. Ron Hubbard, who took the basic concept of techniques to purge confusions from the mind, replaced the substance with a bunch of attractive flim-flam, and founded Scientology. And like Hubbard’s superstar followers, many of this century’s most successful people have been notably irrational.

There seems to me to be approximately zero empirical evidence that x-rationality has a large effect on your practical success, and some anecdotal empirical evidence against it. The evidence in favor of the proposition right now seems to be its sheer obviousness. Rationality is the study of knowing the truth and making good decisions. How the heck could knowing more than everyone else and making better decisions than them not make you more successful?!?

The shortform response to this is that the people who are successful at things by being ‘rationalist-y’ about them usually don’t call what they do rationality. Bruce Lee did not call his style ‘rationality’, but his description of it could be quoted in The Sequences:

I have not invented a “new style,” composite, modified or otherwise that is set within distinct form as apart from “this” method or “that” method. On the contrary, I hope to free my followers from clinging to styles, patterns, or molds. Remember that Jeet Kune Do is merely a name used, a mirror in which to see “ourselves”. . . Jeet Kune Do is not an organized institution that one can be a member of. Either you understand or you don’t, and that is that. There is no mystery about my style. My movements are simple, direct and non-classical. The extraordinary part of it lies in its simplicity. Every movement in Jeet Kune Do is being so of itself. There is nothing artificial about it. I always believe that the easy way is the right way.

Jeet Kune Do is simply the direct expression of one’s feelings with the minimum of movements and energy. The closer to the true way of Kung Fu, the less wastage of expression there is. Finally, a Jeet Kune Do man who says Jeet Kune Do is exclusively Jeet Kune Do is simply not with it. He is still hung up on his self-closing resistance, in this case anchored down to reactionary pattern, and naturally is still bound by another modified pattern and can move within its limits. He has not digested the simple fact that truth exists outside all molds; pattern and awareness is never exclusive. Again let me remind you Jeet Kune Do is just a name used, a boat to get one across, and once across it is to be discarded and not to be carried on one’s back.

In other words, Bruce Lee follows the way of winning. Famous gamer David Sirlin would not call what he does rationality, but his book Playing To Win has a better definition of rationality than the ‘systematized winning’ found in The Sequences:

You will not see a classic scrub throw his opponent five times in a row. But why not? What if doing so is strategically the sequence of moves that optimizes his chances of winning? Here we’ve encountered our first clash: the scrub is only willing to play to win within his own made-up mental set of rules. These rules can be staggeringly arbitrary. If you beat a scrub by throwing projectile attacks at him, keeping your distance and preventing him from getting near you—that’s cheap. If you throw him repeatedly, that’s cheap, too. We’ve covered that one. If you block for fifty seconds doing no moves, that’s cheap. Nearly anything you do that ends up making you win is a prime candidate for being called cheap. Street Fighter was just one example; I could have picked any competitive game at all.

Doing one move or sequence over and over and over is a tactic close to my heart that often elicits the call of the scrub. This goes right to the heart of the matter: why can the scrub not defeat something so obvious and telegraphed as a single move done over and over? Is he such a poor player that he can’t counter that move? And if the move is, for whatever reason, extremely difficult to counter, then wouldn’t I be a fool for not using that move? The first step in becoming a top player is the realization that playing to win means doing whatever most increases your chances of winning. That is true by definition of playing to win. The game knows no rules of “honor” or of “cheapness.” The game only knows winning and losing.

“Rationality is when you stop living your life by fake rules” is a heuristic I tell others often, it’s beautiful in its succinctness and simple enough that almost anyone can understand it.

In Moneyball, Billy Beane and Bill James did not call what they do rationality, yet Billy’s picks were quite literally done by a stats whiz that studied behavioral economics (read: formal rationality) in college and Bill Jame’s ‘sabermetrics’ (“the empirical analysis of baseball”) community sounds almost like LessWrong in its heyday, with the same pattern of a grumpy founder that quits after everyone he shares his insight with proves to be totally inadequate:

Jame's literary powers combined with his willingness to answer his mail to create a movement. Research scientists at big companies, university professors of physics and economics and life sciences, professional statisticians, Wall Street analysts, bored lawyers, math wizards unable to hold down regular jobs — all these people were soon mailing James their ideas, criticisms, models, and questions. His readership must have been one of the strangest groups of people ever assembled under one idea. Before he found a publisher, James had four readers he considered "celebrities."

…

"I hate to say it and I hope you're not one of them," he wrote in his final, 1988 Baseball Abstract, "but I am encountering more and more of my own readers that I don't even like, nitwits who glom onto something superficial in the book and misunderstand its underlying message … Whereas I used to write one 'Dear Jackass' letter a year, I now write maybe thirty." The growing misunderstanding between himself and his readership was, he felt, not adding to the sum total of pleasure or interest in the universe. "I am no longer certain that the effects of my doing this kind of research are in the best interest of the average baseball fan," he explained. "I would like to pretend that the invasion of statistical gremlins crawling at random all over the telecast of damn near every baseball game is irrelevant to me, that I really had nothing to do with it … I know better. I didn't create this mess, but I helped."

Rationality does in fact seem to work, but the people who actually use it do not generally call themselves ‘rationalists’.

The Social Reality Narrative

The top comment on That Magical Click is by ‘pjeby’, who replies:

One of the things that I’ve noticed about this is that most people do not expect to understand things. For most people, the universe is a mysterious place filled with random events beyond their ability to comprehend or control. Think “guessing the teacher’s password”, but not just in school or knowledge, but about everything.

Such people have no problem with the idea of magic, because everything is magic to them, even science.

An anecdote: once, when I still worked as software developer/department manager in a corporation, my boss was congratulating me on a million dollar project (revenue, not cost) that my team had just turned in precisely on time with no crises.

Well, not congratulating me, exactly. He was saying, “wow, that turned out really well”, and I felt oddly uncomfortable. After getting off the phone, I realized a day or so later that he was talking about it like it was luck, like, “wow, what nice weather we had.”

So I called him back and had a little chat about it. The idea that the project had succeeded because I designed it that way had not occurred to him, and the idea that I had done it by the way I negotiated the requirements in the first place—as opposed to heroic efforts during the project—was quite an eye opener for him.

Fortunately, he (and his boss) were “clicky” enough in other areas (i.e., they didn’t believe computers were magic, for example) that I was able to make the math of what I was doing click for them at that “teachable moment”.

Unfortunately, most people, in most areas of their lives treat everything as magic. They’re not used to being able to understand or control anything but the simplest of things, so it doesn’t occur to them to even try. Instead, they just go along with whatever everybody else is thinking or doing.

For such (most) people, reality is social, rather than something you understand/ control.

Having experienced this when I was younger I think this idea is broadly correct. However as I’ll explain in the rest of this post the fear of death and social reality as barriers to rational thinking are not mutually exclusive. In fact, they reinforce and are deeply entangled with each other.

The Intelligence Narrative

Another common explanation for the small number of rationalists is that rationality requires a level of intelligence that’s very rare. Consider The Unz Review’s The 7 tribes of intellect which says of the top 5% of human intelligence:

These are the top 5%. If you are fortunate enough to be in this category, the world is your oyster, unless you blow it by getting drunk, or by imagining that you are so bright that no further work is required, or you go off the rails into being some sort of clever fool, due to some personality difficulty.

…

They can deal with tasks which require the application of specialised background knowledge, dis-embedding the features of a problem from a text, and drawing high-level inferences from highly complex text with multiple distractors. They can almost certainly do the previous credit card comparison task; they can summarise from a given text two ways in which lawyers may challenge prospective jurors; and, using a calculator, determine the total cost of carpet to cover a room, given the dimensions of the room and the cost per square yard of carpeting. (There you are, at the apotheosis of intellect. You can challenge a juror and carpet a room). Their occupations will include the professions, the sciences and, with experience and application, the top posts in business and government. Entertainments will include most artistic and literary endevours, and theories will be seen as interesting in themselves. Vocabularies are in the 30,000 to 42,000 range, which is probably as high as you can go without using lots of technical terms. In modern welfare states they would be high net contributors, very probably supporting two or even three households in addition to their own, and have property and savings. In IQ terms they are 125 and above.

I’ve sometimes told people that if they don’t know how to use a spreadsheet, ipython, or another tool for quantitative thinking they’re not a rationalist. It naturally follows that intelligence is a bottleneck. The sort of thinking you have to do if you want to be reliably correct about things is hard and even the brightest peoples capacity for it is limited. IQ points don’t really grant magic abilities, they grant modest abilities the average reader of this blog would be surprised to learn most people don’t have. On the 2016 LessWrong Survey the median respondent claimed to have an IQ of 138. Obviously a survey is not necessarily going to get us the most reliable data on this, but I can’t say I really think they’re lying.

At the same time, the intelligence barrier might not be as high as is commonly assumed. In his Superforecasting, Tetlock finds that the average IQ of his best performers in the geopolitical forecasting tournament was 80th percentile, far below the 1:1000 rarity implied by LessWrong and SlateStarCodex survey results. This makes sense exactly because the skills provided by IQ points are so modest. Beyond a certain point, if raw intelligence was the only way to get ahead humanity wouldn’t get very far. Assuming it’s possible to teach advanced epistemics to the merely intelligent rather than exclusively the ultra-intelligent this has serious strategy considerations for handling existential risk and other issues.

The Fear of Death Narrative

In his The Denial of Death anthropologist Ernest Becker identifies the fear of death as the unifying psychological struggle between man and the natural world. Here he is describing the confabulation and resistance to thinking that characterizes most people:

Now these euphemisms mean usually that he accepts to work on becoming the father of himself by abandoning his own project and by giving it over to The Fathers. The castration complex has done its work, and one submits to social reality. He can now deflate his own desires and claims, and can play it safe in the world of the powerful elders. He can even give his body over to the tribe, the state, the embracing magical umbrella of the elders and their symbols, that way it will no longer be a dangerous negation for him. But there is no real difference between a childish impossibility and an adult one. The only thing that the person achieves is a practiced self deceit, what we call the Mature Character.

Take stock of those around you and you will hear them talk in precise terms about themselves and their surroundings. Which would seem to point to them having ideas on the matter. But start to analyze those ideas and you will find that they hardly reflect in any way the reality to which they appear to refer. And if you go deeper you will discover that there is not even an attempt to adjust the ideas to this reality. Quite the contrary, through these notions the individual is trying to cut off any personal vision of reality, of his own very life. For life is at the start a chaos in which one is lost, the individual suspects this but he is frightened at finding himself face to face with this terrible reality and tries to cover it over with a curtain of fantasy where everything is clear. It does not worry him that his ideas are not true. He uses them as trenches for the defenses of his existence, as scarecrows to frighten away reality.

This is not necessarily an intuitive notion. I had previously favored the utility, social reality, and intelligence narratives as the basic explanation for why there were so few rationalists. The fear of death was an important factor, but one that played second fiddle to these more important bottlenecks. Over time though I’ve updated towards a more primal rejection, basic primordial fears in the human animal which encourage poor thinking. Beyond its above average explanatory power, I provide four basic arguments for why we should expect the fear of death to be special even before we dig into any detailed analysis:

• The Argument From Empiricism: Death is an irrationality hotspot, and the denial of death represents perhaps the most brazen example of unsanity which is still tolerated in the modern world. People say in all seriousness that after their body is destroyed, they will be resurrected into another plane of existence or reincarnated on earth. They confabulate the most bizarre metaphysics to support these claims, it’s probably not a coincidence that people’s brains shut off when it comes to cryonics. People do not say “I may not have mated, but after I die my essence will spread out among the living and my bloodline will continue in the tribe.”, they’re very sober about the consequences of not reproducing.

• The Argument From Culture: As Becker points out in his book, and as you’d learn from reading any number of anthropology papers, some form of the denial of death is a cultural universal. Becker is particularly insightful with his notion of an immortality project, by which people use magical thinking to defeat death even in ostensibly secular guises. While we’re all familiar with the ordinary religious forms of death-denial, even supposedly ‘secular’ nations such as the Soviet Union made up for their state sponsored atheism by emphasizing the immortality of living on through one’s industrial or scientific accomplishments. It is notable that by controlling a group immortality project, tribes and societies gain a symbolic control over life and death with which they can magically kill defectors and deviants.

• The Argument From Biography: When telling their life stories rationalists tend to cite facing the reality of death as a key moment of development. Alfred Korzybski for example actually wrote his Manhood of Humanity, a book whose thesis was that the mismatch between the growth curve in technology and the growth curve in civilizing capability would inevitably lead to X-Risk, before he wrote his famous Science and Sanity which founded General Semantics. The impetus for that was his participation in WWI, which forced him to seriously consider the question of how to prevent such horrible wars in the future. Eliezer Yudkowsky has his Coming Of Age sequence where he discusses the realization that there is no magic that stops really bad things from happening. My own journey started with the childhood realization that there is nothing to stop my world from being ended by a nuclear war. It was my experience trying to explain my terror to family, friends, and available adults that led me to instantly sympathize with HJPEV when he explains his own battle against ordinary unsanity about risk.

• The Argument From Reflection: If you get into Buddhist meditation or acid, you’ll eventually find that the layers of your identity begin to peel away. At the bottom of your motivation stack you find the fear of death, which to continue living you must leave alone and reattach to the material world. Someone else following the same procedure of “take acid once a week and see what happens” reports a similar melting away of inhibitions surrounding the fear of death. For many readers this will be a “pfft, so what” sort of deal, but if people hack their brains to become uninhibited agents and then report the change as having faced the reality of death I think that’s notable.

Map-Territory Errors

In his Denial of Death Becker says that the best interpretation of the observations made by Freud and other early psychoanalysts is they point to a deep human trauma about the fear of death. He posits people create an identity and buy into a social symbol system in large part to ward off the fear of death. This is because humans are, as Alfred Korzybski identified them, symbolic creatures. We are separated from the rest of earthly existence by the ability to bind time, and transmit observations as little pieces of culture. The problem is that the symbolic is the realm of the gods, but people are still mortal and creaturely. As the famous Jewish creation myth goes, man has the Knowledge of higher worlds, and his punishment is to die knowing. In order to continue existing normally with this massive mortal horror hovering over us all the time, is to selectively deny reality.

Why is our selective denial of the reality of death so damaging to our ability to think? The basic reason is that it involves what I call the refusal of phenomenological necessity. Necessity can be stated simply as: 2 and 2 equals 4, conclusions follow from their premises. In order to deny the reality of death, we must also deny the reality of anything which might be able to show us our mortality. This includes of course, basically any system of ordered thinking. Or at least, any system of ordered thinking based on perception of the world around us. Consider the classic syllogism: All men are mortal. Socrates is a man. Therefore, Socrates is mortal. Deep damage has to be done to your thinking to avoid absorbing the importance of logic that simple. Naturally then we should expect most people are failing to absorb the simple logic of many things.

So it’s not surprising that the most consequential map territory errors are motivated by fear. As children we believed there were monsters under our bed, but more extraordinary was the belief we could banish them by hiding under a blanket. This is the fundamental essence of a map-territory error. We control our perceptions, not the variables those perceptions are nominally meant to track. This creates magical beliefs, where we think that controlling our perceptions (the map) alters the territory.

Becker notes that during childhood development, there is usually a stage where the child believes themselves to be omnipotent. When a small child wants something, they signal their desire and the desire is fulfilled. In fact, it is entirely sensible for them to conclude they are omnipotent. Eventually, the child has desires which outstrip what is possible for their caregivers to provide, and the child often wishes for their caretaker’s death. Because the child believes they’re omnipotent, this juvenile death fantasy is taken as a serious threat in their own mind, and they feel extreme guilt about it.

This speaks to the fact that the default is to have deeply magical beliefs about reality. This impulse is so strong, that even after we have (in the abstract) banished any possibility of psi, ghosts, spirits, etc; we still have people insisting that there must be something more, some way in which the map controls the territory. Because it seems so intuitive and compelling to us that it does.

Death and The Roots of Magick

In shamanic practice, the shaman is typically associated with death and the dead. The shaman is a necromancer, a spirit-channeler, a traveler between the world of the living and the world of the infinite cosmology of dead things. Shamans are supposed to gain their powers through a near death experience, or some proxy thereof. Modern day notions of magic are the descendants of these ancient astral spirit guides. Thus the roots of magick and sorcery are in death, and management of the fear of death. We see this in anthropological accounts of sorcery among more primitive societies, take for example this account of a deadly magician:

During my first fieldwork, Asao was the scariest man in the village – a sagguma, and proud of it. People would have openly despised him, only it was too dangerous to do so. It was safer to fear him, and that they certainly did…Sangguma [sorcerers] are said to acquire ghostly powers by mastering magical skills, submitting to harsh bodily disciplines, and drinking the fluids of a rotting corpse. Asao did not simply admit to all of this, he boasted of it. Animal familiars (mostly night birds) spied for him and brought him news of distant places. Asao claimed the ability to fly and to make himself invisible. With ostentatious glee, he told of participating in attacks (sangguma usually work in teams of two or three) on selected victims…Occasionally, he would be mysteriously absent for days or weeks at a time, presumably in retreat to purify his magical powers or on commission to stalk and attack someone in another, possibly distant, place (Tuzin, 57).

What’s striking to me about this is the similarity, many steps removed, from the modern day “edgelord”, whose social role is to be a troll, culture warrior, or contrarian killer of comforting untruths. This is probably not a coincidence. As Becker points out, it is important for people to control social norms and expression, to enforce religious rules because to lose control of these things is to lose their sense of control over life and death. Tribal control of immortality symbols is used as a psychological weapon against would-be magicians, to be a sorcerer then is to be someone who has stepped outside of social reality:

So, what is a sorcerer? A sorcerer is a – real or perceived – violator of norms of conduct. Such atypical behaviors often entail great risk. One who transgresses taboos that are not particularly esteemed, or that indicate one’s impressive abilities, can gain greater status and prestige, while those who infringe on regulations widely considered legitimate earn the enmity of kith and kin. This is the paradox at the heart of sorcery – the sorcerer seizes power or inadvertently orchestrates his own demise, on occasion performing each concurrently.

Something To Protect

Which brings us to another key point of becoming a rationalist: Something To Protect. In his Rationality, Eliezer Yudkowsky writes that the impulse to become a rationalist must come from protecting another being or entity; it can’t come from protecting yourself. I know this is false, because for me it did come from protecting myself. But I think I know why he would believe this. If you are a selfish creature as people fundamentally are, and you think that your beliefs control reality, to acknowledge the reality of death is to kill whatever you gaze at with it. It’s easier then for us to face the reality of death through a proxy than to acknowledge the reality of our own death. The exact mechanics of why it’s easier are tricky. One cynical possibility is that the proxy is sacrificial. To acknowledge the reality of a family member dying or animals dying is to subject them to death, which instantly creates emotional attachment and feelings of fear, guilt, and responsibility. The proxy stands in for us, and lets us see the horror of death without risking our own life being taken by magic.

A more optimistic possibility is that Something To Protect allows you to elevate a thing over your own survival. Having found something you care more about than living, risking magical death is not quite so horrifying in comparison.

Keeping Your Identity Small

This brings us to identity. In Becker’s view, identity is about putting up a wall between yourself and reality.

It’s notable that Korzybski called confusing the layers of abstraction identification. Korzybski saw identity in the sense of Aristotle’s “A is A” as the core obstacle to rationality. He advocated the removal of words such as “I” and “is” from everyday speech. He also felt that the splitting of man into a “spirit” being separate from an animal being was responsible for much philosophical woe. It is only by accepting our embodied, creaturely nature that we can take full advantage of our ability to think according to General Semantics. Accepting our embodied creaturely nature is of course also to accept our mortality, as it is the creaturely aspects of man which make him decay and die.

As Paul Graham notes in his essays, things which are part of our identity are things which we parse as being direct attacks on us when they’re criticized. Conveniently, anything that’s part of your identity you can’t think clearly about (because to think clearly about it would be to be ‘killed’ by magic). If as Becker says our identities exist to protect us from the reality of death (and they at least in part do), then it stands to reason that one of the most powerful interventions to become more rational is to tear down the damn wall.

This is, as far as I can tell, absolutely the case. The most powerful rationality intervention, bar none, is Paul Graham’s simple notion of keeping your identity small (which he is light on the details of how to actually do, but the goal is sound). Really, the average person should just grind flaying off useless or maladaptive identity aspects and then come back and try the other epistemic enhancing techniques. In the same way that say, you might grind Buddhist meditation before going after stoic control of your emotions.

Even if you’re not sold on the fear of death thesis, identity is still probably the place to start for most people. When we let ideas become part of our identity, they become sacred and it’s not possible to update on them even if they’re wrong. This creates natural bottlenecks on the road to developing uncommon sense, which have to be overcome by shedding ad-hoc identity constructs.

Atheism

This also explains another ‘mysterious’ feature of rationality: Why the association with New Atheism? Can’t you be Catholic and be a rationalist, as many practitioners of General Semantics were?

For example Samuel Bois, who wrote the 1966 General Semantics classic The Art of Awareness: A Handbook on Epistemics and General Semantics was Catholic. It’s tempting then to think that we can retain our faith and be a rationalist, but it’s precisely because the denial of death is so damaging to our thinking that we can’t. Until we break down our fake immortality, it’s not really possible to perceive the world clearly.

This theory, that atheism is a fundamental point because it is key to facing the reality of death, is importantly enough more predictive than the idea that atheism is important because you need to reject the uniquely damaging influence of Christianity. Because it needs to be atheism, agnosticism won’t do. Many people in the modern era are agnostic; they have a vague feel good religious apathy which serves to prevent them from having to think very hard about this subject. Those people still act stupid in the way Christians act stupid, even though they’ve rejected Abrahamic faith. Therefore we know that the important feature is facing the reality of death, not rejecting Christ, Yahweh or Allah. In fact even your median atheist has probably still not quite processed the reality of death, they’re just more capable of doing so.

Existential Risk

It also explains the focus on existential risks. With nuclear arms we already have the tools to destroy ourselves, if nukes are not enough we are likely to get tools that are some time during the 21st century. Most people are fundamentally crippled in their ability to think about this; again, to think about it would be to be killed by magic. Therefore we can infer on priors that existential risks are almost certainly underfocused on in proportion to their seriousness and severity. If your society has problems thinking about something, it’s a safe bet that issues involving that topic are not getting the attention or rigor that they deserve.

This is why Eliezer Yudkowsky wrote his Sequences in the first place. The problem was not that people were failing to understand complex concepts, but rather that they were failing to see the simple logic of scary ideas. Once you’ve faced the reality of death and stopped living your life by fake rules, understood Darwin, shed your fake immortality, rooted out animism from your intuitions and learned a bit about thinking clearly concerns about existential risk are straightforward obvious ideas, not arcane cultish nuttery.

Rethinking Rationality Training

If all this is true we should take a hard look at how we’ve been trying to “train” rationality up to this point. More classes on critical thinking won’t help if the barriers are emotion rather than skill based. Below are some ideas to consider.

Keeping Your Identity Small

I recently read a post where the author said they’d had trouble implementing Paul Graham’s Keep Your Identity Small in a useful way. It was only after becoming more confident in their ability to get by without any particular identity feature that they could stop identifying as this or that. I’m honestly a little skeptical, and suspect the reader misunderstood what Graham was trying to get across. So lets take their example: If you’re someone whose only way to connect with other people is jokes, you might think something like:

“I’m a funny person.”

Are you a funny person? Maybe you are, but are you more than that? Are you always a funny person? Using phrases like “I am” or “is” or “be” in the wrong ways can reinforce static self concepts[0]. To change and then stop identifying is to miss the entire point, you were supposed to notice your limitations by not identifying and then change. The author’s insistence that keeping your identity small is not actionable advice frustrates me a little because I know it is.

How I learned to do it was:

1. Read Paul Graham’s essay.
2. Start noticing identity driven defensiveness as a salient feature of other peoples behavior.
3. Eventually notice when I have this feature in my behavior.
4. When I notice, ask “Do I endorse this part of my identity? Is this identity feature maladaptive?”
5. If I endorse it, reinforce/lean in. If I don’t, step back/unfuse.

Map/Territory

In his research Korzybski found that map/territory errors were best trained out of people by having them spend time with a model of the ladder of abstraction[1]. Bruce Kodish describes this in Korzybski: A Biography:

The diagram could be used as a tool to help bring human thinking to the human level. A person could keep it in front of himself as a reference to help distinguish the levels when dealing with any problem (A statement about a descriptive statement - an inference - is not the descriptive statement; a label or description of an object is not the object; the object is not the invisible, inferred process; etc.) In order to time-bind most effectively, a person had to understand and use the mechanism correctly by recognizing and distinguishing the levels or orders in any situation. This was not a statistical approach to a science of man but one based on human potentiality.

I suspect that something like this would help a lot more than any workshop. People need simple things they can practice on their own to get past identity and ontological confusion in their thinking.

Meditation

Many CFAR-ish people claim that meditation practice was essential to them becoming more rationalist, but can’t explain why. This explains why: Meditation practices help you face the reality of death and process it. It stops being a thing that “works but we don’t know why just trust us” and gives us a predictive model of why we should expect it to work, and what things will contribute to it working or not.

Practicing Dying

Supposedly Becker often told people that they should practice dying. I’m not sure exactly what this entailed, but I do a similar thing to help me think about X-Risk. I’ll lay down in bed and imagine that I’m about to die in the next 5-15 minutes. Die of what you ask? Oh, any number of things. The most common is a nuclear war, but it’s often being turned into grey goo by a rampant superintelligence, or more mundane causes of death like cancer or a virus. When I first started doing this I found it very distressing, but over time I’ve gotten a lot more capable at soberly considering the end of my existence.

It’s notable that the rhetoric used to talk about defeating death might actually be having the opposite of its intended effect on many people. Emphasizing the horror and tragedy of death is useful if you’ve already accepted its reality and need social permission to say the obvious. If you’re crippled in your ability to think about mortality however, this rhetoric probably reinforces the flinch reaction people have and takes them farther away from reality.

Bibliography

1. Drive Yourself Sane: Using The Uncommon Sense Of General Semantics, Third Edition (2011) by Susan and Bruce Kodish.

2. Korzybski: A Biography (2011) by Bruce Kodish