Feb 2, 2018
If you’ve been in InfoSec for a while you probably have significant experience with third-party security questionnaires. They’re basically the new firewall. Everyone is asking everyone else if they have one.
I’ve been troubled for years by the whole charade, but I just recently put my finger on the reason.
Third-party security questionnaires are the equivalent of asking someone if they’re an axe murderer.
As it turns out, most people say no to this question—including most axe murderers. So the value in the process seems to be finding the rare sociopath who says yes.
Here’s 95% of security questionnaire interactions reduced to three sentences:
COMPANY: Hey, question for you. Do you put customer data at extreme risk through gross negligence?
VENDOR: Nope.
COMPANY: Whew! I was worried about that. Glad I asked!
Cool assessment, bro.
Basically, nobody has time to do basement inspections, so asking people nicely is the only option we have.
InfoSec departments are becoming security questionnaire farms, passing forms back and forth like currency.
It’s like echo talking to chargen with no actual work getting done.
Most companies have dozens, hundreds, or thousands of vendors that they deal with, and in order to show due diligence they need to ask most of them if they’re doing these special 743 things.
I regularly see 25-75% of security teams’ effort focused on sending and answering these questionnaires. It’s a full-time job, often for multiple people. And if you ask most people on an internal security team if they give honest answers when responding, you’re likely to get aggressive laughter or a red-faced stare at the ground.
Everyone is lying, and everyone knows it.
And here’s the worst part: sometimes you will see an honest response.
Yeah, so, actually, we don’t have backups yet. Or a logging solution. Actually, to be super transparent, we’re in the process of standing all of this up right now.
Wow, refreshing. So you tell the business and they respond back that they really like the product so they’ll accept the risk.
Cool assessment, bro.
The sad truth is that an actual assessment of a company’s security, to really determine if they’re keeping your data safe, is essentially impossible to do for even one vendor if they’re doing their best to hide things from you.
I’ve seen and heard of dozens of examples where an in-depth assessment team—with full support and transparency from management—still takes days or weeks of interviews and technical review to uncover the worst security flaws in a company. All this while industry-leading auditors are also onsite putting their “all clear” stamp on the same company.
So a dedicated team can be onsite for days, with full management and staff support and transparency, and still not find the dead bodies, but we think an outsider sending a form is going to somehow reveal the truth?
It’s fantasy, full stop.
But this isn’t to say that some good doesn’t come out of third party assessments.
If you ask someone if they’re extremely negligent with your data, or if they go on killing sprees when the sun goes down—and they say yes—that is in fact useful information to have. There’s also the PCI effect where companies are trying to improve their posture so that they have to lie less when they respond to the questionnaires.
In this industry we take whatever wins we can get.
As with any security theater, the real problem here is the disconnect between how much security people think they’re getting from security questionnaires vs. how much they’re actually getting. If I had to put a number to it, questionnaires rate at something like a 2 out of 10 in security effectiveness, and many people think they’re getting something like 8 out of 10.
We can’t just stop doing these assessments, but that gap has consequences, and it’s time we start talking honestly about it.
Notes
- There are actually companies that do security questionnaires well, and get significant value from the process, but I contacted both of them and they weren’t able to speak on record.
- An earlier version of this essay appeared on the IOActive blog.
- Many people cite “due diligence” as a form of value in security questionnaires. I agree, but it’s the same due diligence as asking people if they are serial killers. If they say no, what have you really accomplished?
