High-Entropy Writing

Applying Shannon's entropy concept to essays, talks, and presentations

Author

Daniel Miessler
August 07, 2023

I read a post by Derek Sivers recently that reminded me of Claude Shannon’s concept of Entropy. The post was about the his opinion that talks should always be surprising.

People only really learn when they’re surprised. If they’re not surprised, then what you told them just fits in with what they already know. No minds were changed. No new perspective. Just more information.

So my main advice to anyone preparing to give a talk on stage is to cut out everything from your talk that’s not surprising.

Derek Sivers

That pinged my brain super hard because I read a full biography on Claude Shannon a couple of years ago, and it went deep into his invention of information theory. Here’s the basic idea.

Shannon entropy quantifies the amount of information in a message. For example, a message that is always the same contains no new information, so its entropy is zero. On the other hand, a message that is always different contains a lot of new information, so its entropy is high.

GPT-4

“New” information

Both Sivers’ post and entropy orbit the same concept, which is “new information”.

I love that. It makes me happy.

So from now on I’m going to ask myself when I make a piece of content, “Is this content High-Entropy?” If not, maybe I should do what Sivers says and cut the stuff that isn’t new.

Surprise is new++

But that’s not quite it. In Shannon’s work, and in Sivers’ post, the idea went beyond just new. It was about what was surprising.

Maybe surprise is like high-end newness. It’s like new, but with a pop.

Maybe that’s the standard. What’s surprising about this piece of content I’m creating? What makes a person say “Wow”? What makes them see things differently?

So maybe I’m looking to make content that’s high in entropy, and specifically high in surprise.

Wait, what about methodologies?

But there’s a problem with this model, or at least a question. Are the world’s best talks like this? Are there counter-examples?

Is there any room for something delivered perfectly but that isn’t surprising? I can’t think of one.

Wait, how about a framework that takes really good ideas and turns them into a structure that’s useful, transferable, and repeatable? How about a great checklist?

In that case you have known items just arranged into something useful, right? Where’s the surprise there?

Maybe the surprise is the elegance of the structure. It’s clarity. It’s usability. It’s conciseness.

Yeah, I think that’s worth some surprise, even if the content itself is a set of known items.

Surprise it is

So I’m going with it.

High-entropy content. High in surprise.

And of course it still has to be structured and delivered well, but in this model, “well” means in a way that emphasizes the surprising bits.