I’ve Been Too Sloppy in the Past

sloppy data

I’ve needed to write this post for around 10 years or so. I guess today is the day.

Far too much writing on this site has been too loose with claims—especially going back to 2000, 2005, and even 2010. I got much better after 2015, but I continue to improve in this regard and am getting close to my desired state.

It’s not easy for me to write this post because I wish I hadn’t made these mistakes.

Anyway, what am I talking about? I’m talking about posts like this one, where I talk about the separation of classes in the Bay Area and elsewhere in the world.

I basically start with an idea—which I absolutely still believe by the way—that starts out pretty cool. It’s basically an argument and admonition that if you’re not preparing your children, friends, and loved ones to become part of the upper class, then you will default into the lower one. I then go on to describe how those two classes are different, and how I see this playing out in the future.

Fine. No problem.

The problem is that I came up with my own estimates for percentages of the current upper class percentages, and what they might turn into in the future. And I made pretty visuals showing those numbers.

the-bifurcation-of-america

An example of what not to do when pontificating about numbers

That’s lame.

Now, in my defense, I did say right there in the visual that they were estimates. And I also said they were my own estimates explicitly in the text. And I also put it again in the footnotes. But this was 2017, and I’ve doubtlessly done worse before 2017.

Either way, it’s gross to take your own conjecture about future numbers—which nobody can possibly know—and make them visually appealing in a way that looks like a high-quality data visualization. The key term there is data. People see high-quality visualizations with precise numbers and they assume high-quality data sources. Period. End of story.

Now, I do reserve the right to pontificate. To theorize. To think out loud. To wonder. To just riff on ideas. I can and will do that, without apology. So there is some part of that post, and others like it, that I will continue to defend.

Unsupervised Learning — Security, Tech, and AI in 10 minutes…

Get a weekly breakdown of what's happening in security and tech—and why it matters.

The problem is when I invisibly and imperceptibly—even to myself—make a voice and tone pivot into presenting as if the content is already established fact, backed by data. That’s gross, and inexcusable. And that’s not something I wish to have associated with me or this site.

In various cleanings over the last 20 years, I’ve deleted over 4,000 posts for being too short or low in quality.

So, I want to clearly apologize to the universe for having done that in the past. And when I come across posts where I’ve done so I’ll be cleaning them up or deleting them.

And I’ll continue to struggle with the line between free, fearless thought, pondering, and speculation…vs. authoritative dissemination of information. This problem is magnified by the fact that I also put out technical content that I am careful to be accurate with. So people see that content and then see me exploring some random idea and they get confused.

That’s my fault, not theirs.

Even if a careful reading would reveal the distinction, they shouldn’t have to read that carefully to find it. I should simply stop writing in a way that produces that confusion.

Happily, a number of mentors have told me that I’ve improved in this in recent years. I think it’s just because I read so much more now, and I see how it’s possible to use different types of voices to pass on different types of information—whether that’s your own ponderings or solid facts.

And so the journey continues. If you’ve read this far it’s probably because you like some of my content at least some of the time. And I thank you for that. 104%.

Notes

  1. Thanks to my new friend who I met today who reminded me I needed to write this post.

Related posts: