The word singularity gets thrown around constantly right now. Most of the people using it would struggle to tell you what it means.

It gets used as a synonym for AGI. Also for ASI. Also for "the day AI gets really good," or "the day the robots take over," or just a vague wave at some future where things are weird.

The real meaning is more specific than any of those, and more interesting.

That's it. It's a claim about us.

Where the word comes from ​

Most people think Kurzweil. The Singularity Is Near came out in 2005 and sold a lot of copies.

He was about fifty years late.

The first use I know of is from 1958. Stanislaw Ulam wrote a tribute to John von Neumann after he died, and he was remembering their conversations:

One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue. Stanislaw Ulam, Tribute to John von Neumann, 1958

The end is the part that matters. Beyond which human affairs, as we know them, could not continue. That's a much bigger claim than technology speeding up.

Von Neumann was pointing at a horizon, decades before anyone could have meant AI by it.

The best definition is Vinge's ​

Vernor Vinge taught math and computer science at San Diego State. He also wrote science fiction. In 1993 he gave a paper at a NASA symposium called The Coming Technological Singularity, and that's where this stops being a passing remark and turns into an argument.

His opening is about as blunt as it gets:

Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended. Vernor Vinge, The Coming Technological Singularity, 1993

But here's the line that matters, and the reason he picked that word:

In physics, a singularity is where your equations stop giving answers. Vinge is saying the same thing happens to our models of the future once something smarter than us is doing the driving. Your predictions stop coming out at all.

I like this a lot, because it's a claim about us. It's an admission that we go blind.

One more thing about Vinge's version. He listed four ways superhuman intelligence might show up, and AI is only the first two:

1. Computers that "wake up" and are superhumanly intelligent
2. Large computer networks, together with their users, waking up as a single intelligent entity
3. Human and computer interfaces getting so intimate that the user is effectively superhuman
4. Biology finding ways to improve the human intellect directly

Three of those still have humans in them. That's a wider frame than the one that survived.

Why it would happen fast ​

There's a third name here, and he's the one who explains the mechanism.

In 1965 I.J. Good, a statistician who worked with Turing at Bletchley Park, described an "ultraintelligent machine." A machine good enough at design to build a better machine than itself. Which builds a better one. Which builds a better one.

That's why people expect this to arrive all at once. The loop feeds itself.

So: von Neumann saw the horizon. Good explained how we'd get there fast. Vinge said what it actually is.

My definitions of AGI and ASI ​

Here are the ones I've settled on. I go deeper on them in My Updated Definitions of AGI vs. ASI.

Both of those describe a machine. You could put one on a bench and measure it.

How they fit together ​

The singularity works differently. AGI and ASI are about the machine. The singularity is about us, and specifically about the moment we lose the ability to see what's coming.

Which means you can have one without the other.

You can have AGI and no singularity. I think this is the part people miss. Something as good as a human expert at every cognitive role would rearrange the entire economy, and we could still reason about it, because we already know what human experts do. It's a world with a billion more professionals in it. Enormous. Also forecastable.

ASI is where it breaks. Once something can do any cognitive role better than any human ever has, it stops being a tool you point at problems and starts being the thing that decides. Predicting what a smarter thing decides would require being as smart as it.

Why I like this ​

It's one of the coolest ideas in the space, and it gets flattened into a buzzword constantly.

Knowing where it came from helps, because the idea predates the current hype cycle by decades. Von Neumann was talking about it in the fifties. Vinge wrote it up in 1993. Both of them were just thinking out loud.

And knowing it's a different thing from AGI or ASI changes what you're arguing about. When someone says we're close to the singularity, the real question is whether we're near the point where our models of the future stop working. Benchmark scores say very little about that.

That's a different question. It's also a lot more unsettling.

Summary ​

1. People use the singularity as a synonym for AGI and ASI, and it means something different from both
2. The word goes back to 1958, in Ulam's tribute to von Neumann, describing a point "beyond which human affairs, as we know them, could not continue"
3. Vinge gave it the definition worth keeping, in 1993: "It is a point where our models must be discarded and a new reality rules"
4. I.J. Good supplied the mechanism in 1965 with the ultraintelligent machine that designs its own successor, which is why we'd expect it to hit suddenly
5. AGI and ASI describe a machine's capability. The singularity describes our own blindness. Different kind of claim entirely
6. The main thing: the singularity is the point past which we can't really predict anything, because at that point it's essentially ASIs running things