Moloch: The Most Dangerous Idea

August 2021

Moloch is a strong candidate for the most dangerous idea. If we’re alone in this universe, it’s possibly because of Moloch.

It’s pronounced MOL-uck.

What is it? Practically speaking, it’s something like an accelerating race towards a goal that has both a tremendous payoff and that guarantees our destruction. It’s a race to the bottom, where the bottom is the destruction of our species.

Moloch is a race that’ll probably kill us, but that we can’t help but run.

I heard Max Tegmark talk about the concept recently on Lex Fridman’s podcast, and have heard the term a few times elsewhere in the last few months. But never before that, which is curious.

Wikipedia’s Article

The concept ( מלך mlk in Hebrew) comes from the Old Testament as a name of a Canaanite god associated with human sacrifice, but some scholars also think it might refer to sacrifice itself. Either way it was considered a bad thing in the Bible wherever it was mentioned.

That definition also fits well with the modern interpretation. Sacrifices were done to gain something from the gods, so it’s essentially, “doing something bad so that you can get something in return”.

Another interpretation is one that Max mentioned in the podcast with Lex, which is a demon that convinces you to play this game. So in that model it’s not the game itself that’s the enemy, but the force that’s enticing us to play the game.

There are remarkably few definitions available anywhere, so this is my attempt at it. I’m not sure that distinction matters much, so I’m going to refine the definition above and say it’s:

The inevitable game

As Max talks about on the podcast, Moloch is toxically seductive. It stacks selfishness on top of FOMO on top of practicality.

Here are some examples:

There are smaller-scale moloch games as well.

• The Race to AGI

• Nuclear Armament

• Human Cloning

• Gene Editing

• Growing Economies Using Fossil Fuels

Taking the game of the hour as an example, everyone is sprinting towards AGI at top speed not only because they want the benefits—which they do—but also because they know everyone else is sprinting. So that’s the FOMO aspect.

The worst part is that everyone knows they should stop running, but they can’t.

But we also know it’ll give us tremendous economic and military advantages. So the greed kicks in and makes us run even faster.

And finally we have good ol’ self-preservation. Living under a super-intelligence-powered China would not be fun for anyone. And that’s assuming China could control it.

So even if the West was able to control its greed, and wanted to stop, we know China and Russia won’t. So we won’t either.

The Hail Mary solution

Individuals can’t stop the race because you need everyone to stop at the same time.

The depressing part is how difficult the solution is. It’s obvious. It’s right there in front of us. But nobody is going to do it.

We have to all stop together.

The US. China. Russia. Europe. Asia. The entire world has to stop together and say,

But we probably won’t do that—hence why this is probably the most dangerous idea in the world. Maybe it’s this Moloch Barrier that stops most civilizations from progressing beyond a certain point in their development.

All those trillions of stars out there—all those planets—and we haven’t seen anyone. Maybe they’re just too far away. Some experts believe that. But maybe billions of civilizations got to where we are and they just couldn’t make it past the Moloch Barrier.

Maybe they couldn’t find a way to exit the race before they killed themselves with AI, nukes, bioweapons, or whatever. We need to find a way out of this race.

Nothing is more important.

Notes

1. Tegmark points out some examples of anti-molochs, including gossip, the legal system, and regulation.

2. Howl, by Allen Ginsberg is a poem about Moloch. MORE